Sep 04 22:43:11.341052 ip-10-0-128-32 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 04 22:43:11.341061 ip-10-0-128-32 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 04 22:43:11.341068 ip-10-0-128-32 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 04 22:43:11.341217 ip-10-0-128-32 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 04 22:43:21.346810 ip-10-0-128-32 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 04 22:43:21.346827 ip-10-0-128-32 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 44295611f5d04caeb833a60a927cd973 --
Sep 04 22:45:11.790936 ip-10-0-128-32 systemd[1]: Starting Kubernetes Kubelet...
Sep 04 22:45:12.198515 ip-10-0-128-32 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 22:45:12.198515 ip-10-0-128-32 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 04 22:45:12.198515 ip-10-0-128-32 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 22:45:12.198515 ip-10-0-128-32 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 04 22:45:12.198515 ip-10-0-128-32 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 22:45:12.200604 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.200518    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 04 22:45:12.206114 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206099    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 22:45:12.206114 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206114    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206117    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206121    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206124    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206128    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206131    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206134    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206137    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206140    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206143    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206145    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206148    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206150    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206153    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206156    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206159    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206169    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206172    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206175    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206178    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 22:45:12.206178 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206181    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206183    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206186    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206189    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206192    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206195    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206198    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206200    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206203    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206206    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206208    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206211    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206213    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206216    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206219    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206222    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206224    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206227    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206229    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206232    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 22:45:12.206768 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206234    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206237    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206240    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206242    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206244    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206247    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206249    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206252    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206254    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206257    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206260    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206262    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206265    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206268    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206271    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206274    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206277    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206279    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206282    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206284    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 22:45:12.207581 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206287    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206289    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206292    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206294    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206297    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206299    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206302    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206304    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206307    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206310    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206313    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206315    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206318    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206321    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206323    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206326    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206329    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206331    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206334    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206336    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 22:45:12.208212 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206339    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206342    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206347    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206351    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206354    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206358    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206361    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206364    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206368    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206371    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206374    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206377    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206381    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206385    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206388    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206391    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206394    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.206397    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208386    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 22:45:12.208726 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208400    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208404    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208407    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208410    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208412    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208415    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208418    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208421    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208424    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208427    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208429    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208432    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208435    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208437    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208440    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208442    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208445    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208449    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208452    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 22:45:12.209186 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208456    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208459    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208462    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208464    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208467    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208470    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208472    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208475    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208478    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208480    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208489    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208493    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208498    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208502    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208505    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208508    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208511    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208513    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208516    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208519    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 22:45:12.209715 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208522    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208524    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208527    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208529    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208532    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208535    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208537    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208540    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208543    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208546    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208548    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208551    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208554    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208557    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208560    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208562    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208565    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208567    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208570    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 22:45:12.210213 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208573    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208575    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208577    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208580    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208582    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208585    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208588    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208591    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208594    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208596    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208599    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208601    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208604    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208607    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208609    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208612    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208615    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208617    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208619    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208622    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 22:45:12.210697 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208624    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208627    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208629    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208632    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208659    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208662    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208665    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208667    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208670    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208672    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208675    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208678    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208680    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208687    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208690    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208692    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208696    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208700    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208704    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 22:45:12.211181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.208707    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209153    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209158    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209161    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209164    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209167    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209171    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209174    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209177    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209180    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209184    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209187    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209190    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209193    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209195    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209198    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209201    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209209    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209212    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209215    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 22:45:12.211663 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209217    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209220    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209222    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209225    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209227    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209230    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209232    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209235    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209237    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209240    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209242    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209246    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209248    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209251    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209254    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209256    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209258    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209261    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209264    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209268    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 22:45:12.212165 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209270    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209273    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209276    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209279    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209281    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209284    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209287    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209289    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209292    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209294    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209303    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209306    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209309    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209311    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209314    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209317    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209320    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209322    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209325    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209327    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 22:45:12.212673 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209330    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209337    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209340    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209343    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209346    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209348    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209351    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209354    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209356    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209358    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209361    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209364    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209367    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209369    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209372    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209374    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209377    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209379    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209382    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 22:45:12.213161 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209384    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209387    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209389    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209392    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209402    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209406    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209408    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209411    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209414    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209416    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209419    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209421    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209424    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209426    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209430    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209433    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209436    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209439    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209442    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 22:45:12.213618 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209444    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.209447    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209526    2578 flags.go:64] FLAG: --address="0.0.0.0"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209558    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209566    2578 flags.go:64] FLAG: --anonymous-auth="true"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209571    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209575    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209578    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209582    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209586    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209590    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209593    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209596    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209600    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209603    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209606    2578 flags.go:64] FLAG: --cgroup-root=""
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209609    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209612    2578 flags.go:64] FLAG: --client-ca-file=""
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209622    2578 flags.go:64] FLAG: --cloud-provider="external"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209625    2578 flags.go:64] FLAG: --cluster-dns="[]"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209632    2578 flags.go:64] FLAG: --cluster-domain=""
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209635    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209652    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209655    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 04 22:45:12.214089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209658    2578 flags.go:64] FLAG: --container-log-max-files="5"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209662    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209665    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209670    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209673    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209676    2578 flags.go:64] FLAG: --contention-profiling="false"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209679    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209682    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209686    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209688    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209693    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209696    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209698    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209701    2578 flags.go:64] FLAG: --enable-load-reader="false"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209704    2578 flags.go:64] FLAG: --enable-server="true"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209707    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209714    2578 flags.go:64] FLAG: --event-burst="100"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209718    2578 flags.go:64] FLAG: --event-qps="50"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209721    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209724    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209727    2578 flags.go:64] FLAG: --eviction-hard=""
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209731    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209733    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209737    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209739    2578 flags.go:64] FLAG: --eviction-soft=""
Sep 04 22:45:12.214791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209742    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209745    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209754    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209757    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209759    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209762    2578 flags.go:64] FLAG: --fail-swap-on="true"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209766    2578 flags.go:64] FLAG: --feature-gates=""
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209770    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209773    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209776    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209780    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209782    2578 flags.go:64] FLAG: --healthz-port="10248"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209785    2578 flags.go:64] FLAG: --help="false"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209788    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209791    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209794    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209797    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209801    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209804    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209807    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209810    2578 flags.go:64] FLAG: --image-service-endpoint=""
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209812    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209815    2578 flags.go:64] FLAG: --kube-api-burst="100"
Sep 04 22:45:12.215390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209818    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209821    2578 flags.go:64] FLAG: --kube-api-qps="50"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209824    2578 flags.go:64] FLAG: --kube-reserved=""
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209827    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209829    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209832    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209835    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209838    2578 flags.go:64] FLAG: --lock-file=""
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209841    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209843    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209846    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209852    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209864    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209867    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209871    2578 flags.go:64] FLAG: --logging-format="text"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209873    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209877    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209880    2578 flags.go:64] FLAG: --manifest-url=""
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209882    2578 flags.go:64] FLAG: --manifest-url-header=""
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209887    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209890    2578 flags.go:64] FLAG: --max-open-files="1000000"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209895    2578 flags.go:64] FLAG: --max-pods="110"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209898    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209901    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209903    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 04 22:45:12.216015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209907    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209910    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209913    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209916    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209924    2578 flags.go:64] FLAG: --node-status-max-images="50"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209927    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209929    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209932    2578 flags.go:64] FLAG: --pod-cidr=""
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209935    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209941    2578 flags.go:64] FLAG: --pod-manifest-path=""
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209943    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209947    2578 flags.go:64] FLAG: --pods-per-core="0"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209950    2578 flags.go:64] FLAG: --port="10250"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209953    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209955    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e90f2f68033e6fd8"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209958    2578 flags.go:64] FLAG: --qos-reserved=""
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209962    2578 flags.go:64] FLAG: --read-only-port="10255"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209965    2578 flags.go:64] FLAG: --register-node="true"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209968    2578 flags.go:64] FLAG: --register-with-taints=""
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209972    2578 flags.go:64] FLAG: --registry-burst="10"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209980    2578 flags.go:64] FLAG: --registry-qps="5"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209984    2578 flags.go:64] FLAG: --reserved-cpus=""
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209987    2578 flags.go:64] FLAG: --reserved-memory=""
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209991    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209994    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 04 22:45:12.216632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.209997    2578 flags.go:64] FLAG: --rotate-certificates="false"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210000    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210003    2578 flags.go:64] FLAG: --runonce="false"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210006    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210009    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210011    2578 flags.go:64] FLAG: --seccomp-default="false"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210014    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210017    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210020    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210023    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210026    2578 flags.go:64] FLAG: --storage-driver-password="root"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210029    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210032    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210034    2578 flags.go:64] FLAG: --storage-driver-user="root"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210037    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210040    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210043    2578 flags.go:64] FLAG: --system-cgroups=""
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210046    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210051    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210054    2578 flags.go:64] FLAG: --tls-cert-file=""
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210057    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210063    2578 flags.go:64] FLAG: --tls-min-version=""
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210066    2578 flags.go:64] FLAG: --tls-private-key-file=""
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210068    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210071    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 04 22:45:12.217298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210074    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210077    2578 flags.go:64] FLAG: --v="2"
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210081    2578 flags.go:64] FLAG: --version="false"
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210087    2578 flags.go:64] FLAG: --vmodule=""
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210090    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.210094    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210200    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210204    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210207    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210210    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210213    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210216    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210219    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210222    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210225    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210227    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210230    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210233    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210236    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210239    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210241    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 22:45:12.217974 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210244    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210246    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210249    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210252    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210254    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210257    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210260    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210263    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210265    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210268    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210271    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210273    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210276    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210280    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210285    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210289    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210291    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210295    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210299    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 22:45:12.218500 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210302    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210307    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210310    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210312    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210315    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210318    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210333    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210339    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210343    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210346    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210349    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210352    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210354    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210357    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210360    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210363    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210366    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210368    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210371    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210374    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 22:45:12.218991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210377    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210379    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210382    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210384    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210387    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210390    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210393    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210396    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210399    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210402    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210405    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210408    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210410    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210414    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210417    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210419    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210422    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210424    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210427    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210430    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 22:45:12.219493 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210432    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210435    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210437    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210440    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210443    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210445    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210448    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210450    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210453    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210455    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210458    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210460    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210463    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210466    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210468    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210471    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210474    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210478    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210480    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210486    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 22:45:12.220017 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210489    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210491    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210495    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210497    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.210500    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.211180    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.218610    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.218625    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.218659    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.218665    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218726    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218733    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218737    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218741    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 22:45:12.220514 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218744    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218747    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218750    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218753    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218756    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218758    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218761    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218764    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218766    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218769    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218772    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218774    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218777    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218780    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218782    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218785    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218787    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218790    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218793    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218795    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 22:45:12.220878 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218799    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218801    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218804    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218806    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218809    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218813    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218816    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218818    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218822    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218826    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218830    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218832    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218835    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218838    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218840    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218843    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218846    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218849    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218851    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 22:45:12.221401 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218854    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218856    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218859    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218862    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218864    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218867    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218870    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218872    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218875    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218878    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218881    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218884    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218886    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218889    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218893    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218895    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218898    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218901    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218904    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218907    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 22:45:12.221916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218911    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218914    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218917    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218920    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218923    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218926    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218929    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218932    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218935    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218937    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218940    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218943    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218945    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218948    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218951    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218953    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218956    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218959    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218961    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218964    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 22:45:12.222457 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218967    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218970    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218972    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218975    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218978    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218980    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218983    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218986    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218988    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218991    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218994    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.218997    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219000    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219003    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219005    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 22:45:12.222980 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219008    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.219013    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219126    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219130    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219133    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219135    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219138    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219141    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219144    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219147    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219149    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219152    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219155    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219157    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219160    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 22:45:12.223346 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219163    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219165    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219168    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219170    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219173    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219176    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219178    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219181    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219183    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219186    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219189    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219192    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219194    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219197    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219199    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219202    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219205    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219207    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219210    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219212    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 22:45:12.223735 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219215    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219218    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219220    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219223    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219225    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219228    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219231    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219233    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219236    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219239    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219241    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219244    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219247    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219249    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219252    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219254    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219257    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219260    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219262    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219265    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 22:45:12.224224 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219267    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219270    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219272    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219275    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219278    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219281    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219283    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219286    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219288    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219291    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219293    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219297    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219301    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219305    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219308    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219312    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219316    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219320    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219323    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 22:45:12.224724 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219326    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219329    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219332    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219334    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219337    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219340    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219343    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219346    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219348    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219351    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219353    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219356    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219358    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219361    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219364    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219366    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219369    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219372    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219375    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219377    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 22:45:12.225181 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219380    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 22:45:12.225734 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219382    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 22:45:12.225734 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219385    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 22:45:12.225734 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219387    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 22:45:12.225734 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219390    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 22:45:12.225734 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219392    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 22:45:12.225734 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:12.219395    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 22:45:12.225734 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.219400    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 22:45:12.225734 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.219498    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 04 22:45:12.225734 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.221819    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 04 22:45:12.225734 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.222835    2578 server.go:1019] "Starting client certificate rotation"
Sep 04 22:45:12.225734 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.222944    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 04 22:45:12.225734 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.222975    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 04 22:45:12.226675 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.226661    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 04 22:45:12.228026 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.228000    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 04 22:45:12.236304 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.236284    2578 log.go:25] "Validated CRI v1 runtime API"
Sep 04 22:45:12.239704 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.239687    2578 log.go:25] "Validated CRI v1 image API"
Sep 04 22:45:12.242544 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.242528    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 04 22:45:12.245912 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.245889    2578 fs.go:135] Filesystem UUIDs: map[6d9d6167-b15c-4001-a2a2-700142109940:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d96b198c-2da4-4322-aef8-383ed2e0e545:/dev/nvme0n1p3]
Sep 04 22:45:12.245977 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.245908    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 04 22:45:12.249539 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.249521    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 04 22:45:12.253091 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.252966    2578 manager.go:216] Machine: {Timestamp:2026-09-04 22:45:12.251290933 +0000 UTC m=+0.361411531 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107912 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec259a9c793c000340b214bb9556ac38 SystemUUID:ec259a9c-793c-0003-40b2-14bb9556ac38 BootID:44295611-f5d0-4cae-b833-a60a927cd973 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:55:a2:1f:e4:0d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:55:a2:1f:e4:0d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:45:dd:bc:3f:9a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 04 22:45:12.253091 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.253086    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 04 22:45:12.253209 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.253197    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 04 22:45:12.256944 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.256916    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 04 22:45:12.257073 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.256947    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-32.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 04 22:45:12.258535 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.258524    2578 topology_manager.go:138] "Creating topology manager with none policy"
Sep 04 22:45:12.258574 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.258538    2578 container_manager_linux.go:309] "Creating device plugin manager"
Sep 04 22:45:12.258574 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.258546    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 04 22:45:12.259266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.259255    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 04 22:45:12.260987 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.260977    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 04 22:45:12.265485 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.265473    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 04 22:45:12.265612 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.265602    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 04 22:45:12.265655 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.265633    2578 kubelet.go:486] "Attempting to sync node with API server"
Sep 04 22:45:12.265693 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.265657    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 04 22:45:12.265693 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.265677    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 04 22:45:12.265693 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.265686    2578 kubelet.go:398] "Adding apiserver pod source"
Sep 04 22:45:12.265791 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.265696    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 04 22:45:12.266020 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.266008    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 04 22:45:12.266074 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.266028    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 04 22:45:12.268353 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.268336    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 04 22:45:12.271181 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.271166    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 04 22:45:12.271249 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.271196    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 04 22:45:12.271364 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.271354    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 04 22:45:12.271406 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.271370    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 04 22:45:12.271406 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.271385    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 04 22:45:12.271406 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.271391    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 04 22:45:12.271406 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.271399    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 04 22:45:12.271406 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.271405    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 04 22:45:12.272055 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.272036    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 04 22:45:12.272055 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.272049    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 04 22:45:12.272055 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.272059    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 04 22:45:12.272206 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.272064    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 04 22:45:12.272206 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.272082    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 04 22:45:12.272206 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.272090    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 04 22:45:12.272206 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.272086    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-564h7"
Sep 04 22:45:12.273814 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.273804    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 04 22:45:12.273814 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.273814    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 04 22:45:12.276587 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.276551    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-32.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 04 22:45:12.276708 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.276674    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 04 22:45:12.277848 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.277836    2578 server.go:1268] "Started kubelet"
Sep 04 22:45:12.277939 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.277916    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 04 22:45:12.278012 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.277966    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 04 22:45:12.278049 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.278035    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 04 22:45:12.278591 ip-10-0-128-32 systemd[1]: Started Kubernetes Kubelet.
Sep 04 22:45:12.283141 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.283107    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 04 22:45:12.283346 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.283323    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-564h7"
Sep 04 22:45:12.284677 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.284630    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 04 22:45:12.285343 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.285319    2578 server.go:310] "Adding debug handlers to kubelet server"
Sep 04 22:45:12.288651 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.288623    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 04 22:45:12.288651 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.288634    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 04 22:45:12.288781 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.288735    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:12.289260 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.288871    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-32.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 04 22:45:12.289431 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.289416    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 04 22:45:12.289558 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.289540    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 04 22:45:12.289558 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.289525    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 04 22:45:12.289709 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.289513    2578 factory.go:55] Registering systemd factory
Sep 04 22:45:12.289709 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.289580    2578 factory.go:223] Registration of the systemd container factory successfully
Sep 04 22:45:12.289709 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.289629    2578 reconstruct.go:97] "Volume reconstruction finished"
Sep 04 22:45:12.289709 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.289670    2578 reconciler.go:29] "Reconciler: start to sync state"
Sep 04 22:45:12.289863 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.289826    2578 factory.go:153] Registering CRI-O factory
Sep 04 22:45:12.289863 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.289836    2578 factory.go:223] Registration of the crio container factory successfully
Sep 04 22:45:12.289941 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.289910    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 04 22:45:12.289941 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.289929    2578 factory.go:103] Registering Raw factory
Sep 04 22:45:12.290031 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.289945    2578 manager.go:1196] Started watching for new ooms in manager
Sep 04 22:45:12.290165 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.288977    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-32.ec2.internal.18d23f95401ca960  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-32.ec2.internal,UID:ip-10-0-128-32.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-32.ec2.internal,},FirstTimestamp:2026-09-04 22:45:12.277805408 +0000 UTC m=+0.387926031,LastTimestamp:2026-09-04 22:45:12.277805408 +0000 UTC m=+0.387926031,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-32.ec2.internal,}"
Sep 04 22:45:12.290330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.290316    2578 manager.go:349] Starting recovery of all containers
Sep 04 22:45:12.294363 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.294339    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 22:45:12.296910 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.296888    2578 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-32.ec2.internal\" not found" node="ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.301848 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.301831    2578 manager.go:354] Recovery completed
Sep 04 22:45:12.307769 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.307753    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:12.311043 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.309879    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:12.311043 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.309909    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:12.311043 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.309925    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:12.311043 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.310851    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 04 22:45:12.311043 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.310861    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 04 22:45:12.311043 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.310879    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 04 22:45:12.311363 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.311272    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:12.312074 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.312056    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:12.312159 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.312085    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:12.312159 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.312095    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:12.313029 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.313016    2578 policy_none.go:49] "None policy: Start"
Sep 04 22:45:12.313074 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.313033    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 04 22:45:12.313074 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.313043    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 04 22:45:12.314933 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.314922    2578 policy_none.go:47] "Start"
Sep 04 22:45:12.369934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.355239    2578 manager.go:337] "Starting Device Plugin manager"
Sep 04 22:45:12.369934 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.355282    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 04 22:45:12.369934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.355294    2578 server.go:87] "Starting device plugin registration server"
Sep 04 22:45:12.369934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.355551    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 04 22:45:12.369934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.355562    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 04 22:45:12.369934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.355684    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 04 22:45:12.369934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.355766    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 04 22:45:12.369934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.355776    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 04 22:45:12.369934 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.356295    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 04 22:45:12.369934 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.356349    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:12.411478 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.411446    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 04 22:45:12.412683 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.412662    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 04 22:45:12.412793 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.412716    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 04 22:45:12.412793 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.412742    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 04 22:45:12.412885 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.412790    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 04 22:45:12.414910 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.414888    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 22:45:12.455910 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.455844    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:12.456780 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.456756    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:12.456861 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.456799    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:12.456861 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.456812    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:12.456861 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.456834    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.464690 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.464671    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.464769 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.464695    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-32.ec2.internal\": node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:12.474320 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.474301    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:12.512857 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.512831    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-32.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal"]
Sep 04 22:45:12.512945 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.512903    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:12.514300 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.514286    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:12.514385 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.514318    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:12.514385 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.514331    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:12.516702 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.516687    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:12.516845 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.516830    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.516894 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.516860    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:12.517415 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.517388    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:12.517415 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.517417    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:12.517540 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.517431    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:12.517540 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.517438    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:12.517540 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.517456    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:12.517540 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.517468    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:12.519748 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.519734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.519846 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.519758    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:12.520436 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.520417    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:12.520506 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.520442    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:12.520506 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.520452    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:12.553245 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.553222    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-32.ec2.internal\" not found" node="ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.557519 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.557503    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-32.ec2.internal\" not found" node="ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.574840 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.574817    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:12.591580 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.591557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9dd58ca56a50716b273632f503970b80-config\") pod \"kube-apiserver-proxy-ip-10-0-128-32.ec2.internal\" (UID: \"9dd58ca56a50716b273632f503970b80\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.591580 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.591580    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/365e88cbc7790ec5fadf0193271ff280-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal\" (UID: \"365e88cbc7790ec5fadf0193271ff280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.591730 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.591599    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/365e88cbc7790ec5fadf0193271ff280-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal\" (UID: \"365e88cbc7790ec5fadf0193271ff280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.675936 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.675894    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:12.692311 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.692281    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9dd58ca56a50716b273632f503970b80-config\") pod \"kube-apiserver-proxy-ip-10-0-128-32.ec2.internal\" (UID: \"9dd58ca56a50716b273632f503970b80\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.692416 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.692319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/365e88cbc7790ec5fadf0193271ff280-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal\" (UID: \"365e88cbc7790ec5fadf0193271ff280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.692416 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.692360    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/365e88cbc7790ec5fadf0193271ff280-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal\" (UID: \"365e88cbc7790ec5fadf0193271ff280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.692416 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.692399    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9dd58ca56a50716b273632f503970b80-config\") pod \"kube-apiserver-proxy-ip-10-0-128-32.ec2.internal\" (UID: \"9dd58ca56a50716b273632f503970b80\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.692416 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.692409    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/365e88cbc7790ec5fadf0193271ff280-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal\" (UID: \"365e88cbc7790ec5fadf0193271ff280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.692560 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.692408    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/365e88cbc7790ec5fadf0193271ff280-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal\" (UID: \"365e88cbc7790ec5fadf0193271ff280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.776769 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.776694    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:12.856209 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.856168    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.859698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:12.859677    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:12.877210 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.877186    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:12.979474 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:12.979427    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:13.086031 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:13.085997    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:13.193571 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:13.193546    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:13.224170 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:13.224147    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 04 22:45:13.224610 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:13.224333    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 04 22:45:13.224610 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:13.224333    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 04 22:45:13.288087 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:13.287886    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 22:45:13.289731 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:13.289698    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-03 22:40:13 +0000 UTC" deadline="2028-05-12 23:11:07.636044124 +0000 UTC"
Sep 04 22:45:13.289731 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:13.289725    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14784h25m54.346322578s"
Sep 04 22:45:13.293914 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:13.293892    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:13.395028 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:13.394953    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:13.435916 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:13.435875    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod365e88cbc7790ec5fadf0193271ff280.slice/crio-ac305db1cf256c5e3941349bd3d07f82c9c86f9a6c0bc09fdc5877f5471e9302 WatchSource:0}: Error finding container ac305db1cf256c5e3941349bd3d07f82c9c86f9a6c0bc09fdc5877f5471e9302: Status 404 returned error can't find the container with id ac305db1cf256c5e3941349bd3d07f82c9c86f9a6c0bc09fdc5877f5471e9302
Sep 04 22:45:13.442474 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:13.442456    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 22:45:13.449528 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:13.449504    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dd58ca56a50716b273632f503970b80.slice/crio-f7b16bbfa7c05e30bd7c3dd760f4242bb726410c372d033dddfe06d35e23a4be WatchSource:0}: Error finding container f7b16bbfa7c05e30bd7c3dd760f4242bb726410c372d033dddfe06d35e23a4be: Status 404 returned error can't find the container with id f7b16bbfa7c05e30bd7c3dd760f4242bb726410c372d033dddfe06d35e23a4be
Sep 04 22:45:13.495315 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:13.495269    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:13.595806 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:13.595766    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:13.696426 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:13.696340    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:13.796685 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:13.796626    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-32.ec2.internal\" not found"
Sep 04 22:45:13.809610 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:13.809590    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 22:45:13.889313 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:13.889283    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:13.901511 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:13.901464    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 04 22:45:13.902535 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:13.902514    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-32.ec2.internal"
Sep 04 22:45:13.910150 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:13.910128    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 04 22:45:14.267908 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.267873    2578 apiserver.go:52] "Watching apiserver"
Sep 04 22:45:14.282832 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.282802    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 04 22:45:14.283381 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.283351    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-9dqmt","openshift-network-operator/iptables-alerter-l4sc8","openshift-ovn-kubernetes/ovnkube-node-znc4n","kube-system/kube-apiserver-proxy-ip-10-0-128-32.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9","openshift-cluster-node-tuning-operator/tuned-57gxp","openshift-image-registry/node-ca-4qbrc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal","openshift-multus/multus-additional-cni-plugins-9vmrt","openshift-network-diagnostics/network-check-target-77nr4","kube-system/global-pull-secret-syncer-t4vfv","kube-system/konnectivity-agent-6748j","openshift-dns/node-resolver-k7nf5","openshift-multus/multus-qdvpv"]
Sep 04 22:45:14.286381 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.286348    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:14.286484 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:14.286459    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:14.289060 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.288550    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k7nf5"
Sep 04 22:45:14.292145 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.291336    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.292145 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.291431    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 04 22:45:14.292145 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.291459    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-55q5t\""
Sep 04 22:45:14.292145 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.291479    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 04 22:45:14.292145 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.291341    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 04 22:45:14.293938 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.293731    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.294304 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.294285    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 04 22:45:14.294452 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.294422    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 04 22:45:14.294547 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.294530    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 04 22:45:14.296457 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.294766    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 04 22:45:14.296457 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.294784    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j6qff\""
Sep 04 22:45:14.296457 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.294787    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 04 22:45:14.296457 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.296121    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.296708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.296574    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 04 22:45:14.296708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.296706    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 04 22:45:14.296841 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.296788    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 04 22:45:14.296841 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.296812    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 04 22:45:14.296939 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.296842    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 04 22:45:14.299825 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.297058    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7p6jb\""
Sep 04 22:45:14.299825 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.299149    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.300069 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.299985    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 04 22:45:14.300069 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.300014    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cbbz8\""
Sep 04 22:45:14.300388 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.300183    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 04 22:45:14.300388 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.300256    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301244    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-slash\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301278    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6g6m\" (UniqueName: \"kubernetes.io/projected/6e5ea01c-a8df-4300-b94b-d4b9b0828846-kube-api-access-b6g6m\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301307    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-multus-socket-dir-parent\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-run-k8s-cni-cncf-io\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301356    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6e5ea01c-a8df-4300-b94b-d4b9b0828846-ovnkube-script-lib\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301379    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-hostroot\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d6465b5-2c07-420d-b17d-a9f524a07220-multus-daemon-config\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301427    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-etc-kubernetes\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fhsm\" (UniqueName: \"kubernetes.io/projected/0d6465b5-2c07-420d-b17d-a9f524a07220-kube-api-access-4fhsm\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301476    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-run-openvswitch\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301499    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-node-log\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-run-ovn-kubernetes\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301549    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301576    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6e5ea01c-a8df-4300-b94b-d4b9b0828846-env-overrides\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301602    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e5ea01c-a8df-4300-b94b-d4b9b0828846-ovn-node-metrics-cert\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.302397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301627    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-etc-openvswitch\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301653    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwprb\" (UniqueName: \"kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb\") pod \"network-check-target-77nr4\" (UID: \"9c9d02b6-4018-4f24-9900-df3d1eadbc59\") " pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301687    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-run-ovn\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6e5ea01c-a8df-4300-b94b-d4b9b0828846-ovnkube-config\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301778    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d6465b5-2c07-420d-b17d-a9f524a07220-cni-binary-copy\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-var-lib-cni-multus\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301841    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7655d1a4-b050-4eea-b049-7c6899295630-tmp-dir\") pod \"node-resolver-k7nf5\" (UID: \"7655d1a4-b050-4eea-b049-7c6899295630\") " pod="openshift-dns/node-resolver-k7nf5"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301865    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-kubelet\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301889    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-multus-cni-dir\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301917    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-var-lib-cni-bin\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301943    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-systemd-units\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301964    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302190    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6qv85\""
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.301967    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-run-systemd\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302336    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-multus-conf-dir\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-system-cni-dir\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-log-socket\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302405    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-cni-bin\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.303550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302414    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 04 22:45:14.304548 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302430    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-cni-netd\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.304548 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302454    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-cnibin\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.304548 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302476    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-var-lib-kubelet\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.304548 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-run-multus-certs\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.304548 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302524    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7655d1a4-b050-4eea-b049-7c6899295630-hosts-file\") pod \"node-resolver-k7nf5\" (UID: \"7655d1a4-b050-4eea-b049-7c6899295630\") " pod="openshift-dns/node-resolver-k7nf5"
Sep 04 22:45:14.304548 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302550    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x29p6\" (UniqueName: \"kubernetes.io/projected/7655d1a4-b050-4eea-b049-7c6899295630-kube-api-access-x29p6\") pod \"node-resolver-k7nf5\" (UID: \"7655d1a4-b050-4eea-b049-7c6899295630\") " pod="openshift-dns/node-resolver-k7nf5"
Sep 04 22:45:14.304548 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302575    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-run-netns\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.304548 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302598    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-var-lib-openvswitch\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.304548 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.302623    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-run-netns\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.304548 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.303251    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.304548 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.303803    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4qbrc"
Sep 04 22:45:14.306433 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.305947    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 04 22:45:14.306433 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.306028    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 04 22:45:14.306433 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.306092    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 04 22:45:14.306433 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.306126    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g4xsh\""
Sep 04 22:45:14.306433 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.306345    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 04 22:45:14.306433 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.306364    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l4sc8"
Sep 04 22:45:14.306433 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.306404    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 04 22:45:14.309765 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.307360    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 04 22:45:14.309765 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.307929    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tcjgq\""
Sep 04 22:45:14.309765 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.309183    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nzd7j\""
Sep 04 22:45:14.309765 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.309242    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 04 22:45:14.309765 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.309468    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 04 22:45:14.310130 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.309795    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:14.310130 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.309925    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 04 22:45:14.310130 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:14.309934    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:14.312877 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.312745    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:14.312877 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:14.312829    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:14.319056 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.319033    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6748j"
Sep 04 22:45:14.322401 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.321812    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 04 22:45:14.322401 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.322034    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jfvms\""
Sep 04 22:45:14.322401 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.322229    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 04 22:45:14.327374 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.327352    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tfkkl"
Sep 04 22:45:14.335130 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.335108    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tfkkl"
Sep 04 22:45:14.391176 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.391150    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 04 22:45:14.403380 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403334    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-run-ovn\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.403530 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6e5ea01c-a8df-4300-b94b-d4b9b0828846-ovnkube-config\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.403530 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403414    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d6465b5-2c07-420d-b17d-a9f524a07220-cni-binary-copy\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.403530 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403445    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-var-lib-kubelet\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.403530 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403464    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-run-ovn\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.403530 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403489    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-cnibin\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.403759 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7655d1a4-b050-4eea-b049-7c6899295630-tmp-dir\") pod \"node-resolver-k7nf5\" (UID: \"7655d1a4-b050-4eea-b049-7c6899295630\") " pod="openshift-dns/node-resolver-k7nf5"
Sep 04 22:45:14.403759 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403580    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-multus-cni-dir\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.403759 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403612    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-modprobe-d\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.403759 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403638    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c5024faf-de05-44a5-8094-e921266e1f18-etc-tuned\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.403759 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403680    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5024faf-de05-44a5-8094-e921266e1f18-tmp\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.403759 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-run-systemd\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.403759 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403740    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-multus-conf-dir\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.404031 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403764    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/407039b1-b5f0-4538-8f92-f277e742b9d6-serviceca\") pod \"node-ca-4qbrc\" (UID: \"407039b1-b5f0-4538-8f92-f277e742b9d6\") " pod="openshift-image-registry/node-ca-4qbrc"
Sep 04 22:45:14.404031 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403796    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-run\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.404031 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403822    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-log-socket\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.404031 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403849    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-cnibin\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.404031 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403907    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-run-systemd\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.404031 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.403991    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-log-socket\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.404347 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-multus-conf-dir\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.404347 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-var-lib-kubelet\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.404347 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404174    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-run-multus-certs\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.404347 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404177    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-multus-cni-dir\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.404347 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404209    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-sysctl-conf\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.404347 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404223    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-var-lib-kubelet\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.404347 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404256    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-run-multus-certs\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.404347 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404280    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-cnibin\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.404347 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404298    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-host\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.404698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404383    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d6465b5-2c07-420d-b17d-a9f524a07220-cni-binary-copy\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.404698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404392    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7655d1a4-b050-4eea-b049-7c6899295630-tmp-dir\") pod \"node-resolver-k7nf5\" (UID: \"7655d1a4-b050-4eea-b049-7c6899295630\") " pod="openshift-dns/node-resolver-k7nf5"
Sep 04 22:45:14.404698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-cni-binary-copy\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.404698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x29p6\" (UniqueName: \"kubernetes.io/projected/7655d1a4-b050-4eea-b049-7c6899295630-kube-api-access-x29p6\") pod \"node-resolver-k7nf5\" (UID: \"7655d1a4-b050-4eea-b049-7c6899295630\") " pod="openshift-dns/node-resolver-k7nf5"
Sep 04 22:45:14.404698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404504    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-var-lib-openvswitch\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.404698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404533    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-var-lib-openvswitch\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.404698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404607    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-run-k8s-cni-cncf-io\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.404698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404634    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6e5ea01c-a8df-4300-b94b-d4b9b0828846-ovnkube-config\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.404698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404661    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef-agent-certs\") pod \"konnectivity-agent-6748j\" (UID: \"b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef\") " pod="kube-system/konnectivity-agent-6748j"
Sep 04 22:45:14.405095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-etc-selinux\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.405095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404710    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-run-k8s-cni-cncf-io\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.405095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404733    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:14.405095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-hostroot\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.405095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404799    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-hostroot\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.405095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fhsm\" (UniqueName: \"kubernetes.io/projected/0d6465b5-2c07-420d-b17d-a9f524a07220-kube-api-access-4fhsm\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.405095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404849    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-sysconfig\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.405095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.404978    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e59abefb-388f-4bd6-a8a6-7db9e73010c2-host-slash\") pod \"iptables-alerter-l4sc8\" (UID: \"e59abefb-388f-4bd6-a8a6-7db9e73010c2\") " pod="openshift-network-operator/iptables-alerter-l4sc8"
Sep 04 22:45:14.405095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-node-log\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.405095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405049    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.405095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405072    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-node-log\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.405095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6e5ea01c-a8df-4300-b94b-d4b9b0828846-env-overrides\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405134    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-lib-modules\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405145    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405164    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-etc-openvswitch\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405188    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-etc-openvswitch\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405190    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-var-lib-cni-multus\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8cq2\" (UniqueName: \"kubernetes.io/projected/407039b1-b5f0-4538-8f92-f277e742b9d6-kube-api-access-v8cq2\") pod \"node-ca-4qbrc\" (UID: \"407039b1-b5f0-4538-8f92-f277e742b9d6\") " pod="openshift-image-registry/node-ca-4qbrc"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405262    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-sysctl-d\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405297    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-var-lib-cni-multus\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405302    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-kubelet\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405351    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-var-lib-cni-bin\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405384    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-sys\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405403    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-var-lib-cni-bin\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405410    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-systemd-units\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405353    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-kubelet\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405478    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-systemd-units\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.405649 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405526    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e59abefb-388f-4bd6-a8a6-7db9e73010c2-iptables-alerter-script\") pod \"iptables-alerter-l4sc8\" (UID: \"e59abefb-388f-4bd6-a8a6-7db9e73010c2\") " pod="openshift-network-operator/iptables-alerter-l4sc8"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnrvk\" (UniqueName: \"kubernetes.io/projected/e59abefb-388f-4bd6-a8a6-7db9e73010c2-kube-api-access-rnrvk\") pod \"iptables-alerter-l4sc8\" (UID: \"e59abefb-388f-4bd6-a8a6-7db9e73010c2\") " pod="openshift-network-operator/iptables-alerter-l4sc8"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405523    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6e5ea01c-a8df-4300-b94b-d4b9b0828846-env-overrides\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405586    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-system-cni-dir\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405623    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-cni-bin\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-cni-netd\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405696    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-device-dir\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-cni-bin\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405724    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp5k7\" (UniqueName: \"kubernetes.io/projected/c5024faf-de05-44a5-8094-e921266e1f18-kube-api-access-vp5k7\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405729    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-system-cni-dir\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405756    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7655d1a4-b050-4eea-b049-7c6899295630-hosts-file\") pod \"node-resolver-k7nf5\" (UID: \"7655d1a4-b050-4eea-b049-7c6899295630\") " pod="openshift-dns/node-resolver-k7nf5"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405758    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-cni-netd\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405781    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-run-netns\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-run-netns\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405860    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef-konnectivity-ca\") pod \"konnectivity-agent-6748j\" (UID: \"b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef\") " pod="kube-system/konnectivity-agent-6748j"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405882    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-run-netns\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-kubernetes\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.406266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405930    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2tm8\" (UniqueName: \"kubernetes.io/projected/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-kube-api-access-h2tm8\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405937    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-host-run-netns\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405801    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7655d1a4-b050-4eea-b049-7c6899295630-hosts-file\") pod \"node-resolver-k7nf5\" (UID: \"7655d1a4-b050-4eea-b049-7c6899295630\") " pod="openshift-dns/node-resolver-k7nf5"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405975    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-slash\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.405998    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-slash\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406002    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6g6m\" (UniqueName: \"kubernetes.io/projected/6e5ea01c-a8df-4300-b94b-d4b9b0828846-kube-api-access-b6g6m\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-multus-socket-dir-parent\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406060    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/407039b1-b5f0-4538-8f92-f277e742b9d6-host\") pod \"node-ca-4qbrc\" (UID: \"407039b1-b5f0-4538-8f92-f277e742b9d6\") " pod="openshift-image-registry/node-ca-4qbrc"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406088    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-sys-fs\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406129    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tnhs\" (UniqueName: \"kubernetes.io/projected/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-kube-api-access-7tnhs\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406210    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6e5ea01c-a8df-4300-b94b-d4b9b0828846-ovnkube-script-lib\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d6465b5-2c07-420d-b17d-a9f524a07220-multus-daemon-config\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406262    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-etc-kubernetes\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406365    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-etc-kubernetes\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406263    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d6465b5-2c07-420d-b17d-a9f524a07220-multus-socket-dir-parent\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.406979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406520    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-registration-dir\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406590    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v62bl\" (UniqueName: \"kubernetes.io/projected/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-kube-api-access-v62bl\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-run-openvswitch\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406639    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-run-ovn-kubernetes\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406706    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-kubelet-config\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406686    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-run-openvswitch\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406754    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-dbus\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406767    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e5ea01c-a8df-4300-b94b-d4b9b0828846-host-run-ovn-kubernetes\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406787    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-socket-dir\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-systemd\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406816    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-system-cni-dir\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406841    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406865    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406926    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e5ea01c-a8df-4300-b94b-d4b9b0828846-ovn-node-metrics-cert\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.406973    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwprb\" (UniqueName: \"kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb\") pod \"network-check-target-77nr4\" (UID: \"9c9d02b6-4018-4f24-9900-df3d1eadbc59\") " pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.407287    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d6465b5-2c07-420d-b17d-a9f524a07220-multus-daemon-config\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.407708 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.407318    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6e5ea01c-a8df-4300-b94b-d4b9b0828846-ovnkube-script-lib\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.408518 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.407781    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 04 22:45:14.412071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.412008    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e5ea01c-a8df-4300-b94b-d4b9b0828846-ovn-node-metrics-cert\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.414249 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:14.414227    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:14.414449 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:14.414254    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:14.414449 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:14.414287    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rwprb for pod openshift-network-diagnostics/network-check-target-77nr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:14.415268 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:14.414361    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb podName:9c9d02b6-4018-4f24-9900-df3d1eadbc59 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:14.914342289 +0000 UTC m=+3.024462870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rwprb" (UniqueName: "kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb") pod "network-check-target-77nr4" (UID: "9c9d02b6-4018-4f24-9900-df3d1eadbc59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:14.418087 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.418056    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x29p6\" (UniqueName: \"kubernetes.io/projected/7655d1a4-b050-4eea-b049-7c6899295630-kube-api-access-x29p6\") pod \"node-resolver-k7nf5\" (UID: \"7655d1a4-b050-4eea-b049-7c6899295630\") " pod="openshift-dns/node-resolver-k7nf5"
Sep 04 22:45:14.420048 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.419997    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-32.ec2.internal" event={"ID":"9dd58ca56a50716b273632f503970b80","Type":"ContainerStarted","Data":"f7b16bbfa7c05e30bd7c3dd760f4242bb726410c372d033dddfe06d35e23a4be"}
Sep 04 22:45:14.420906 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.420855    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6g6m\" (UniqueName: \"kubernetes.io/projected/6e5ea01c-a8df-4300-b94b-d4b9b0828846-kube-api-access-b6g6m\") pod \"ovnkube-node-znc4n\" (UID: \"6e5ea01c-a8df-4300-b94b-d4b9b0828846\") " pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.421188 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.421165    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fhsm\" (UniqueName: \"kubernetes.io/projected/0d6465b5-2c07-420d-b17d-a9f524a07220-kube-api-access-4fhsm\") pod \"multus-qdvpv\" (UID: \"0d6465b5-2c07-420d-b17d-a9f524a07220\") " pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.422225 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.422203    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal" event={"ID":"365e88cbc7790ec5fadf0193271ff280","Type":"ContainerStarted","Data":"ac305db1cf256c5e3941349bd3d07f82c9c86f9a6c0bc09fdc5877f5471e9302"}
Sep 04 22:45:14.508297 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508258    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef-konnectivity-ca\") pod \"konnectivity-agent-6748j\" (UID: \"b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef\") " pod="kube-system/konnectivity-agent-6748j"
Sep 04 22:45:14.508483 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-kubernetes\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.508483 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508444    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2tm8\" (UniqueName: \"kubernetes.io/projected/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-kube-api-access-h2tm8\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:14.508590 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508488    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/407039b1-b5f0-4538-8f92-f277e742b9d6-host\") pod \"node-ca-4qbrc\" (UID: \"407039b1-b5f0-4538-8f92-f277e742b9d6\") " pod="openshift-image-registry/node-ca-4qbrc"
Sep 04 22:45:14.508590 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508513    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-sys-fs\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.508590 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508539    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tnhs\" (UniqueName: \"kubernetes.io/projected/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-kube-api-access-7tnhs\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.508590 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508551    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-kubernetes\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.508818 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508610    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-sys-fs\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.508818 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508679    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/407039b1-b5f0-4538-8f92-f277e742b9d6-host\") pod \"node-ca-4qbrc\" (UID: \"407039b1-b5f0-4538-8f92-f277e742b9d6\") " pod="openshift-image-registry/node-ca-4qbrc"
Sep 04 22:45:14.508818 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508698    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.508818 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508745    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:14.508818 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508777    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-registration-dir\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.508818 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508805    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v62bl\" (UniqueName: \"kubernetes.io/projected/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-kube-api-access-v62bl\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-kubelet-config\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508860    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-dbus\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:14.508871    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508886    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-socket-dir\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508892    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef-konnectivity-ca\") pod \"konnectivity-agent-6748j\" (UID: \"b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef\") " pod="kube-system/konnectivity-agent-6748j"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508892    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-registration-dir\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508910    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-systemd\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:14.508941    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret podName:075c9d8c-8c3d-4509-aca6-fa687f5eb1bb nodeName:}" failed. No retries permitted until 2026-09-04 22:45:15.008920585 +0000 UTC m=+3.119041153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret") pod "global-pull-secret-syncer-t4vfv" (UID: "075c9d8c-8c3d-4509-aca6-fa687f5eb1bb") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508960    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-kubelet-config\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508965    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-systemd\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.508964    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-system-cni-dir\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509010    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509037    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-dbus\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509040    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509079    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509092    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-var-lib-kubelet\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.509155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509119    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-cnibin\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509146    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-modprobe-d\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509169    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c5024faf-de05-44a5-8094-e921266e1f18-etc-tuned\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509186    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-socket-dir\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509194    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5024faf-de05-44a5-8094-e921266e1f18-tmp\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509222    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/407039b1-b5f0-4538-8f92-f277e742b9d6-serviceca\") pod \"node-ca-4qbrc\" (UID: \"407039b1-b5f0-4538-8f92-f277e742b9d6\") " pod="openshift-image-registry/node-ca-4qbrc"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509222    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-system-cni-dir\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509228    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-cnibin\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-run\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509266    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-var-lib-kubelet\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509276    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-sysctl-conf\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509316    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509232    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509407    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-sysctl-conf\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509461    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-run\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509470    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-modprobe-d\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509489    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-host\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.510148 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509517    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-cni-binary-copy\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509548    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef-agent-certs\") pod \"konnectivity-agent-6748j\" (UID: \"b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef\") " pod="kube-system/konnectivity-agent-6748j"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-etc-selinux\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509609    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-host\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509630    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-sysconfig\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e59abefb-388f-4bd6-a8a6-7db9e73010c2-host-slash\") pod \"iptables-alerter-l4sc8\" (UID: \"e59abefb-388f-4bd6-a8a6-7db9e73010c2\") " pod="openshift-network-operator/iptables-alerter-l4sc8"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509689    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/407039b1-b5f0-4538-8f92-f277e742b9d6-serviceca\") pod \"node-ca-4qbrc\" (UID: \"407039b1-b5f0-4538-8f92-f277e742b9d6\") " pod="openshift-image-registry/node-ca-4qbrc"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e59abefb-388f-4bd6-a8a6-7db9e73010c2-host-slash\") pod \"iptables-alerter-l4sc8\" (UID: \"e59abefb-388f-4bd6-a8a6-7db9e73010c2\") " pod="openshift-network-operator/iptables-alerter-l4sc8"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509722    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-etc-selinux\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509778    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-sysconfig\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:14.509812    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:14.509862    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs podName:a7ec67b5-ff79-49d1-b063-8cd7c4a56c84 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:15.009845584 +0000 UTC m=+3.119966165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs") pod "network-metrics-daemon-9dqmt" (UID: "a7ec67b5-ff79-49d1-b063-8cd7c4a56c84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-lib-modules\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509943    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8cq2\" (UniqueName: \"kubernetes.io/projected/407039b1-b5f0-4538-8f92-f277e742b9d6-kube-api-access-v8cq2\") pod \"node-ca-4qbrc\" (UID: \"407039b1-b5f0-4538-8f92-f277e742b9d6\") " pod="openshift-image-registry/node-ca-4qbrc"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509971    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-sysctl-d\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.509999    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-sys\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.511071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.510028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.511951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.510055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e59abefb-388f-4bd6-a8a6-7db9e73010c2-iptables-alerter-script\") pod \"iptables-alerter-l4sc8\" (UID: \"e59abefb-388f-4bd6-a8a6-7db9e73010c2\") " pod="openshift-network-operator/iptables-alerter-l4sc8"
Sep 04 22:45:14.511951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.510083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnrvk\" (UniqueName: \"kubernetes.io/projected/e59abefb-388f-4bd6-a8a6-7db9e73010c2-kube-api-access-rnrvk\") pod \"iptables-alerter-l4sc8\" (UID: \"e59abefb-388f-4bd6-a8a6-7db9e73010c2\") " pod="openshift-network-operator/iptables-alerter-l4sc8"
Sep 04 22:45:14.511951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.510111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-device-dir\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.511951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.510134    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp5k7\" (UniqueName: \"kubernetes.io/projected/c5024faf-de05-44a5-8094-e921266e1f18-kube-api-access-vp5k7\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.511951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.510155    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-cni-binary-copy\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.511951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.510156    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-etc-sysctl-d\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.511951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.510356    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-sys\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.511951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.510701    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-device-dir\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.511951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.510829    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5024faf-de05-44a5-8094-e921266e1f18-lib-modules\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.511951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.510815    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.511951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.510963    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e59abefb-388f-4bd6-a8a6-7db9e73010c2-iptables-alerter-script\") pod \"iptables-alerter-l4sc8\" (UID: \"e59abefb-388f-4bd6-a8a6-7db9e73010c2\") " pod="openshift-network-operator/iptables-alerter-l4sc8"
Sep 04 22:45:14.512503 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.512368    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c5024faf-de05-44a5-8094-e921266e1f18-etc-tuned\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.512556 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.512496    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5024faf-de05-44a5-8094-e921266e1f18-tmp\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.512680 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.512660    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef-agent-certs\") pod \"konnectivity-agent-6748j\" (UID: \"b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef\") " pod="kube-system/konnectivity-agent-6748j"
Sep 04 22:45:14.519365 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.519294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2tm8\" (UniqueName: \"kubernetes.io/projected/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-kube-api-access-h2tm8\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:14.519569 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.519539    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tnhs\" (UniqueName: \"kubernetes.io/projected/a6dfb16b-b631-4bce-8d9f-4d307e58dff2-kube-api-access-7tnhs\") pod \"aws-ebs-csi-driver-node-zzxd9\" (UID: \"a6dfb16b-b631-4bce-8d9f-4d307e58dff2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.519685 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.519588    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v62bl\" (UniqueName: \"kubernetes.io/projected/aa4adc0d-b12a-4f86-addd-5c39a9fc1255-kube-api-access-v62bl\") pod \"multus-additional-cni-plugins-9vmrt\" (UID: \"aa4adc0d-b12a-4f86-addd-5c39a9fc1255\") " pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.520179 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.519913    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8cq2\" (UniqueName: \"kubernetes.io/projected/407039b1-b5f0-4538-8f92-f277e742b9d6-kube-api-access-v8cq2\") pod \"node-ca-4qbrc\" (UID: \"407039b1-b5f0-4538-8f92-f277e742b9d6\") " pod="openshift-image-registry/node-ca-4qbrc"
Sep 04 22:45:14.520577 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.520558    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp5k7\" (UniqueName: \"kubernetes.io/projected/c5024faf-de05-44a5-8094-e921266e1f18-kube-api-access-vp5k7\") pod \"tuned-57gxp\" (UID: \"c5024faf-de05-44a5-8094-e921266e1f18\") " pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.520656 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.520563    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnrvk\" (UniqueName: \"kubernetes.io/projected/e59abefb-388f-4bd6-a8a6-7db9e73010c2-kube-api-access-rnrvk\") pod \"iptables-alerter-l4sc8\" (UID: \"e59abefb-388f-4bd6-a8a6-7db9e73010c2\") " pod="openshift-network-operator/iptables-alerter-l4sc8"
Sep 04 22:45:14.603242 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.603206    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 22:45:14.606243 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.606216    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k7nf5"
Sep 04 22:45:14.621073 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.621052    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:14.630078 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.630058    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qdvpv"
Sep 04 22:45:14.638988 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.638967    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9"
Sep 04 22:45:14.645848 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.645831    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-57gxp"
Sep 04 22:45:14.654730 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.654710    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9vmrt"
Sep 04 22:45:14.663627 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.663608    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4qbrc"
Sep 04 22:45:14.673482 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.673460    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l4sc8"
Sep 04 22:45:14.679320 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.679297    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6748j"
Sep 04 22:45:14.749309 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:14.749280    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 22:45:15.013514 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:15.013467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:15.013718 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:15.013633    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:15.013718 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:15.013670    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:15.013718 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:15.013707    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwprb\" (UniqueName: \"kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb\") pod \"network-check-target-77nr4\" (UID: \"9c9d02b6-4018-4f24-9900-df3d1eadbc59\") " pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:15.013905 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:15.013749    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs podName:a7ec67b5-ff79-49d1-b063-8cd7c4a56c84 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:16.013730964 +0000 UTC m=+4.123851546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs") pod "network-metrics-daemon-9dqmt" (UID: "a7ec67b5-ff79-49d1-b063-8cd7c4a56c84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:15.013905 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:15.013777    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:15.013905 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:15.013828    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:15.013905 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:15.013849    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:15.013905 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:15.013854    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret podName:075c9d8c-8c3d-4509-aca6-fa687f5eb1bb nodeName:}" failed. No retries permitted until 2026-09-04 22:45:16.013832823 +0000 UTC m=+4.123953409 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret") pod "global-pull-secret-syncer-t4vfv" (UID: "075c9d8c-8c3d-4509-aca6-fa687f5eb1bb") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:15.013905 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:15.013863    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rwprb for pod openshift-network-diagnostics/network-check-target-77nr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:15.013905 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:15.013906    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb podName:9c9d02b6-4018-4f24-9900-df3d1eadbc59 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:16.013893648 +0000 UTC m=+4.124014232 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwprb" (UniqueName: "kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb") pod "network-check-target-77nr4" (UID: "9c9d02b6-4018-4f24-9900-df3d1eadbc59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:15.338600 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:15.338547    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-03 22:40:14 +0000 UTC" deadline="2028-02-05 15:57:10.174739035 +0000 UTC"
Sep 04 22:45:15.338600 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:15.338592    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12449h11m54.836150498s"
Sep 04 22:45:15.413337 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:15.413304    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:15.413534 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:15.413435    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:15.523991 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:15.523083    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7655d1a4_b050_4eea_b049_7c6899295630.slice/crio-32f8afec4dc20d9854b4ed774a13a9e08cc4496b7a99ad895b46867be1d71800 WatchSource:0}: Error finding container 32f8afec4dc20d9854b4ed774a13a9e08cc4496b7a99ad895b46867be1d71800: Status 404 returned error can't find the container with id 32f8afec4dc20d9854b4ed774a13a9e08cc4496b7a99ad895b46867be1d71800
Sep 04 22:45:15.526865 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:15.526775    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode59abefb_388f_4bd6_a8a6_7db9e73010c2.slice/crio-5f48e964695f051a2b7c6c4a8b251e2e3fc042634cc0da803367130846aa5f3b WatchSource:0}: Error finding container 5f48e964695f051a2b7c6c4a8b251e2e3fc042634cc0da803367130846aa5f3b: Status 404 returned error can't find the container with id 5f48e964695f051a2b7c6c4a8b251e2e3fc042634cc0da803367130846aa5f3b
Sep 04 22:45:15.530692 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:15.530565    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5024faf_de05_44a5_8094_e921266e1f18.slice/crio-2c4fb4c3d235803f6bcf48af3d5a16f6c3c45a2dee1caaeea52d64d6d49f0ff0 WatchSource:0}: Error finding container 2c4fb4c3d235803f6bcf48af3d5a16f6c3c45a2dee1caaeea52d64d6d49f0ff0: Status 404 returned error can't find the container with id 2c4fb4c3d235803f6bcf48af3d5a16f6c3c45a2dee1caaeea52d64d6d49f0ff0
Sep 04 22:45:15.531285 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:15.531266    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d6465b5_2c07_420d_b17d_a9f524a07220.slice/crio-67acf1c1f7c281973987760ffc2eebecc382b39bb430adad6e250e681f5fe649 WatchSource:0}: Error finding container 67acf1c1f7c281973987760ffc2eebecc382b39bb430adad6e250e681f5fe649: Status 404 returned error can't find the container with id 67acf1c1f7c281973987760ffc2eebecc382b39bb430adad6e250e681f5fe649
Sep 04 22:45:15.533444 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:15.533116    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6dfb16b_b631_4bce_8d9f_4d307e58dff2.slice/crio-1ee3b263818efb06ca1ec18ea0206d5446edf15adec67638552e2faf0d65820c WatchSource:0}: Error finding container 1ee3b263818efb06ca1ec18ea0206d5446edf15adec67638552e2faf0d65820c: Status 404 returned error can't find the container with id 1ee3b263818efb06ca1ec18ea0206d5446edf15adec67638552e2faf0d65820c
Sep 04 22:45:15.536646 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:15.536086    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e5ea01c_a8df_4300_b94b_d4b9b0828846.slice/crio-538c89276cdea5207e41f459f728e8e4745f7c5368ae77e50062f7847901c809 WatchSource:0}: Error finding container 538c89276cdea5207e41f459f728e8e4745f7c5368ae77e50062f7847901c809: Status 404 returned error can't find the container with id 538c89276cdea5207e41f459f728e8e4745f7c5368ae77e50062f7847901c809
Sep 04 22:45:15.539015 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:15.537585    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa4adc0d_b12a_4f86_addd_5c39a9fc1255.slice/crio-88c1594442872aab64e355174382a8278cba19257576c01e2d61f4b81b27ab5d WatchSource:0}: Error finding container 88c1594442872aab64e355174382a8278cba19257576c01e2d61f4b81b27ab5d: Status 404 returned error can't find the container with id 88c1594442872aab64e355174382a8278cba19257576c01e2d61f4b81b27ab5d
Sep 04 22:45:15.539264 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:15.539230    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3de408b_3ab1_4bf9_9898_2ca1b2ccd8ef.slice/crio-2e981cc8ebfc01d7c4d42d9bb93d0103e643b2027bdfe288c02456b26c2f579c WatchSource:0}: Error finding container 2e981cc8ebfc01d7c4d42d9bb93d0103e643b2027bdfe288c02456b26c2f579c: Status 404 returned error can't find the container with id 2e981cc8ebfc01d7c4d42d9bb93d0103e643b2027bdfe288c02456b26c2f579c
Sep 04 22:45:16.022258 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.022217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:16.022445 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.022281    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwprb\" (UniqueName: \"kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb\") pod \"network-check-target-77nr4\" (UID: \"9c9d02b6-4018-4f24-9900-df3d1eadbc59\") " pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:16.022445 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.022333    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:16.022530 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:16.022470    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:16.022578 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:16.022538    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs podName:a7ec67b5-ff79-49d1-b063-8cd7c4a56c84 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:18.022518018 +0000 UTC m=+6.132638600 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs") pod "network-metrics-daemon-9dqmt" (UID: "a7ec67b5-ff79-49d1-b063-8cd7c4a56c84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:16.022764 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:16.022734    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:16.022877 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:16.022844    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret podName:075c9d8c-8c3d-4509-aca6-fa687f5eb1bb nodeName:}" failed. No retries permitted until 2026-09-04 22:45:18.022823416 +0000 UTC m=+6.132944007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret") pod "global-pull-secret-syncer-t4vfv" (UID: "075c9d8c-8c3d-4509-aca6-fa687f5eb1bb") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:16.022954 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:16.022764    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:16.022954 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:16.022904    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:16.022954 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:16.022919    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rwprb for pod openshift-network-diagnostics/network-check-target-77nr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:16.023112 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:16.022991    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb podName:9c9d02b6-4018-4f24-9900-df3d1eadbc59 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:18.02297623 +0000 UTC m=+6.133096798 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwprb" (UniqueName: "kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb") pod "network-check-target-77nr4" (UID: "9c9d02b6-4018-4f24-9900-df3d1eadbc59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:16.339908 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.339790    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-03 22:40:14 +0000 UTC" deadline="2028-05-21 20:41:35.853343802 +0000 UTC"
Sep 04 22:45:16.339908 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.339857    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14997h56m19.513492098s"
Sep 04 22:45:16.415375 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.414837    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:16.415375 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:16.414975    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:16.415924 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.415755    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:16.415924 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:16.415875    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:16.433440 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.433401    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-32.ec2.internal" event={"ID":"9dd58ca56a50716b273632f503970b80","Type":"ContainerStarted","Data":"5848ccc7e1e10f527401c4c3b25731f78c221b2fd6339d3d963cd651505b599b"}
Sep 04 22:45:16.438765 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.438736    2578 generic.go:356] "Generic (PLEG): container finished" podID="365e88cbc7790ec5fadf0193271ff280" containerID="cdc18df187f95902c2d11625ba46817e6cdcc91576d1d183ce9c5c8d401428ca" exitCode=0
Sep 04 22:45:16.438877 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.438819    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal" event={"ID":"365e88cbc7790ec5fadf0193271ff280","Type":"ContainerDied","Data":"cdc18df187f95902c2d11625ba46817e6cdcc91576d1d183ce9c5c8d401428ca"}
Sep 04 22:45:16.442961 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.442933    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9vmrt" event={"ID":"aa4adc0d-b12a-4f86-addd-5c39a9fc1255","Type":"ContainerStarted","Data":"88c1594442872aab64e355174382a8278cba19257576c01e2d61f4b81b27ab5d"}
Sep 04 22:45:16.450210 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.450182    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4qbrc" event={"ID":"407039b1-b5f0-4538-8f92-f277e742b9d6","Type":"ContainerStarted","Data":"cd164041edeafc94a51858ef090dfcfddc529c076b9ac2e2520bd7c1aebec9ca"}
Sep 04 22:45:16.463187 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.463087    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9" event={"ID":"a6dfb16b-b631-4bce-8d9f-4d307e58dff2","Type":"ContainerStarted","Data":"1ee3b263818efb06ca1ec18ea0206d5446edf15adec67638552e2faf0d65820c"}
Sep 04 22:45:16.469669 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.469599    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qdvpv" event={"ID":"0d6465b5-2c07-420d-b17d-a9f524a07220","Type":"ContainerStarted","Data":"67acf1c1f7c281973987760ffc2eebecc382b39bb430adad6e250e681f5fe649"}
Sep 04 22:45:16.471594 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.471358    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-32.ec2.internal" podStartSLOduration=2.471338614 podStartE2EDuration="2.471338614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:16.45036956 +0000 UTC m=+4.560490152" watchObservedRunningTime="2026-09-04 22:45:16.471338614 +0000 UTC m=+4.581459204"
Sep 04 22:45:16.480153 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.480117    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6748j" event={"ID":"b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef","Type":"ContainerStarted","Data":"2e981cc8ebfc01d7c4d42d9bb93d0103e643b2027bdfe288c02456b26c2f579c"}
Sep 04 22:45:16.484378 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.484351    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n" event={"ID":"6e5ea01c-a8df-4300-b94b-d4b9b0828846","Type":"ContainerStarted","Data":"538c89276cdea5207e41f459f728e8e4745f7c5368ae77e50062f7847901c809"}
Sep 04 22:45:16.486894 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.486868    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-57gxp" event={"ID":"c5024faf-de05-44a5-8094-e921266e1f18","Type":"ContainerStarted","Data":"2c4fb4c3d235803f6bcf48af3d5a16f6c3c45a2dee1caaeea52d64d6d49f0ff0"}
Sep 04 22:45:16.489606 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.489580    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l4sc8" event={"ID":"e59abefb-388f-4bd6-a8a6-7db9e73010c2","Type":"ContainerStarted","Data":"5f48e964695f051a2b7c6c4a8b251e2e3fc042634cc0da803367130846aa5f3b"}
Sep 04 22:45:16.492363 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:16.492340    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k7nf5" event={"ID":"7655d1a4-b050-4eea-b049-7c6899295630","Type":"ContainerStarted","Data":"32f8afec4dc20d9854b4ed774a13a9e08cc4496b7a99ad895b46867be1d71800"}
Sep 04 22:45:17.413647 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:17.413385    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:17.414163 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:17.413794    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:17.508882 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:17.508797    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal" event={"ID":"365e88cbc7790ec5fadf0193271ff280","Type":"ContainerStarted","Data":"48155c04d6bf9dc5ef6fddb4197c3910fb6e0dedb29d95d5efbe75a2c851fc3e"}
Sep 04 22:45:18.044567 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:18.044490    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:18.044785 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:18.044675    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:18.044785 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:18.044761    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret podName:075c9d8c-8c3d-4509-aca6-fa687f5eb1bb nodeName:}" failed. No retries permitted until 2026-09-04 22:45:22.044734087 +0000 UTC m=+10.154854660 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret") pod "global-pull-secret-syncer-t4vfv" (UID: "075c9d8c-8c3d-4509-aca6-fa687f5eb1bb") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:18.045511 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:18.045099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwprb\" (UniqueName: \"kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb\") pod \"network-check-target-77nr4\" (UID: \"9c9d02b6-4018-4f24-9900-df3d1eadbc59\") " pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:18.045511 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:18.045153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:18.045511 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:18.045273    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:18.045511 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:18.045322    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs podName:a7ec67b5-ff79-49d1-b063-8cd7c4a56c84 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:22.045306372 +0000 UTC m=+10.155426944 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs") pod "network-metrics-daemon-9dqmt" (UID: "a7ec67b5-ff79-49d1-b063-8cd7c4a56c84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:18.045511 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:18.045408    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:18.045511 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:18.045422    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:18.045511 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:18.045437    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rwprb for pod openshift-network-diagnostics/network-check-target-77nr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:18.045511 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:18.045475    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb podName:9c9d02b6-4018-4f24-9900-df3d1eadbc59 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:22.045461149 +0000 UTC m=+10.155581731 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwprb" (UniqueName: "kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb") pod "network-check-target-77nr4" (UID: "9c9d02b6-4018-4f24-9900-df3d1eadbc59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:18.415914 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:18.415878    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:18.416417 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:18.416030    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:18.416509 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:18.416444    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:18.416572 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:18.416536    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:19.413328 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:19.413286    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:19.413595 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:19.413429    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:20.415841 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:20.415807    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:20.416353 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:20.415951    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:20.416353 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:20.416326    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:20.416660 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:20.416432    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:21.413873 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:21.413840    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:21.414081 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:21.413995    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:22.080280 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:22.080234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwprb\" (UniqueName: \"kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb\") pod \"network-check-target-77nr4\" (UID: \"9c9d02b6-4018-4f24-9900-df3d1eadbc59\") " pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:22.080799 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:22.080330    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:22.080799 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:22.080395    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:22.080799 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:22.080446    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:22.080799 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:22.080473    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:22.080799 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:22.080486    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rwprb for pod openshift-network-diagnostics/network-check-target-77nr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:22.080799 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:22.080546    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb podName:9c9d02b6-4018-4f24-9900-df3d1eadbc59 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:30.080529768 +0000 UTC m=+18.190650354 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwprb" (UniqueName: "kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb") pod "network-check-target-77nr4" (UID: "9c9d02b6-4018-4f24-9900-df3d1eadbc59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:22.080799 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:22.080544    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:22.080799 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:22.080544    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:22.080799 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:22.080590    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret podName:075c9d8c-8c3d-4509-aca6-fa687f5eb1bb nodeName:}" failed. No retries permitted until 2026-09-04 22:45:30.080579266 +0000 UTC m=+18.190699847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret") pod "global-pull-secret-syncer-t4vfv" (UID: "075c9d8c-8c3d-4509-aca6-fa687f5eb1bb") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:22.080799 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:22.080653    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs podName:a7ec67b5-ff79-49d1-b063-8cd7c4a56c84 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:30.080614968 +0000 UTC m=+18.190735556 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs") pod "network-metrics-daemon-9dqmt" (UID: "a7ec67b5-ff79-49d1-b063-8cd7c4a56c84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:22.415244 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:22.414689    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:22.415244 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:22.414873    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:22.415244 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:22.414981    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:22.415244 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:22.415096    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:23.413395 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:23.412921    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:23.413395 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:23.413034    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:24.413677 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:24.413626    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:24.414106 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:24.413655    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:24.414106 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:24.413786    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:24.414106 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:24.413904    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:25.413611 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:25.413575    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:25.413813 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:25.413707    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:26.413402 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:26.413371    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:26.413566 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:26.413402    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:26.413566 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:26.413493    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:26.413684 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:26.413655    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:27.413934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:27.413896    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:27.414315 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:27.414009    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:28.413419 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:28.413335    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:28.413586 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:28.413470    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:28.413586 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:28.413525    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:28.413732 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:28.413615    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:29.412955 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:29.412920    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:29.413421 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:29.413049    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:30.140094 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:30.140055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:30.140293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:30.140115    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwprb\" (UniqueName: \"kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb\") pod \"network-check-target-77nr4\" (UID: \"9c9d02b6-4018-4f24-9900-df3d1eadbc59\") " pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:30.140293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:30.140155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:30.140293 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:30.140215    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:30.140293 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:30.140241    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:30.140293 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:30.140274    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:30.140293 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:30.140295    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret podName:075c9d8c-8c3d-4509-aca6-fa687f5eb1bb nodeName:}" failed. No retries permitted until 2026-09-04 22:45:46.140275097 +0000 UTC m=+34.250395661 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret") pod "global-pull-secret-syncer-t4vfv" (UID: "075c9d8c-8c3d-4509-aca6-fa687f5eb1bb") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:30.140578 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:30.140297    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:30.140578 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:30.140314    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rwprb for pod openshift-network-diagnostics/network-check-target-77nr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:30.140578 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:30.140315    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs podName:a7ec67b5-ff79-49d1-b063-8cd7c4a56c84 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:46.140304755 +0000 UTC m=+34.250425319 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs") pod "network-metrics-daemon-9dqmt" (UID: "a7ec67b5-ff79-49d1-b063-8cd7c4a56c84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:30.140578 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:30.140380    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb podName:9c9d02b6-4018-4f24-9900-df3d1eadbc59 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:46.140360138 +0000 UTC m=+34.250480706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwprb" (UniqueName: "kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb") pod "network-check-target-77nr4" (UID: "9c9d02b6-4018-4f24-9900-df3d1eadbc59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:30.413417 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:30.413334    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:30.413877 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:30.413484    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:30.413877 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:30.413527    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:30.413877 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:30.413618    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:31.413351 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:31.413308    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:31.413560 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:31.413439    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:32.414253 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:32.414210    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:32.414726 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:32.414318    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:32.414726 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:32.414372    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:32.414726 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:32.414467    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:33.413735 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:33.413697    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:33.413893 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:33.413831    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:34.414075 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.413923    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:34.414714 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.414025    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:34.414714 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:34.414289    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:34.414714 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:34.414153    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:34.550324 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.550291    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k7nf5" event={"ID":"7655d1a4-b050-4eea-b049-7c6899295630","Type":"ContainerStarted","Data":"4b52662c6eed4c3ed8922b1e3c967ba7b6881c1053659572d408a496bd732c1a"}
Sep 04 22:45:34.552213 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.552190    2578 generic.go:356] "Generic (PLEG): container finished" podID="aa4adc0d-b12a-4f86-addd-5c39a9fc1255" containerID="793a4165f51e4011a49b9cd602057c06103f83b6d8932f5921993ecb90df8fbc" exitCode=0
Sep 04 22:45:34.552317 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.552278    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9vmrt" event={"ID":"aa4adc0d-b12a-4f86-addd-5c39a9fc1255","Type":"ContainerDied","Data":"793a4165f51e4011a49b9cd602057c06103f83b6d8932f5921993ecb90df8fbc"}
Sep 04 22:45:34.554049 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.554028    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4qbrc" event={"ID":"407039b1-b5f0-4538-8f92-f277e742b9d6","Type":"ContainerStarted","Data":"37227389fab2c3deee8f0fc1c304c14d9c2811a28117072ad7940f10e39f167b"}
Sep 04 22:45:34.555659 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.555616    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9" event={"ID":"a6dfb16b-b631-4bce-8d9f-4d307e58dff2","Type":"ContainerStarted","Data":"6f7b6717f8d4d131c07119b6229c82289a113c7a5c911d56cbdebc88ccfa8ef9"}
Sep 04 22:45:34.557161 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.557137    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qdvpv" event={"ID":"0d6465b5-2c07-420d-b17d-a9f524a07220","Type":"ContainerStarted","Data":"93f0b33826196fad59fa6775b05f671797396ffca304713fc90c0aa527614250"}
Sep 04 22:45:34.558633 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.558609    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6748j" event={"ID":"b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef","Type":"ContainerStarted","Data":"3509c16be1d06391c48f9d35cab72459a44911af30389b7702d38c96bf96c470"}
Sep 04 22:45:34.561124 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.561102    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n" event={"ID":"6e5ea01c-a8df-4300-b94b-d4b9b0828846","Type":"ContainerStarted","Data":"d49553bbfbbaaf77bafb94413ba9fc59f5b64b546c6ba7a223480cd9edfd2aac"}
Sep 04 22:45:34.561202 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.561163    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n" event={"ID":"6e5ea01c-a8df-4300-b94b-d4b9b0828846","Type":"ContainerStarted","Data":"12fea5b07eb7db254d100257d4f1cd63d453b2aed58aec443d65a884bc8b7b6e"}
Sep 04 22:45:34.561255 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.561209    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n" event={"ID":"6e5ea01c-a8df-4300-b94b-d4b9b0828846","Type":"ContainerStarted","Data":"818b8aa29623ea7c4f8bef63c5ec188cf6de95a6590935ca4f01f65c42f9c3dd"}
Sep 04 22:45:34.561255 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.561239    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n" event={"ID":"6e5ea01c-a8df-4300-b94b-d4b9b0828846","Type":"ContainerStarted","Data":"8277876eb50eadf65777c105991bccc25864ed9750cc9fbbc4572b2fcd58804c"}
Sep 04 22:45:34.562404 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.562385    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-57gxp" event={"ID":"c5024faf-de05-44a5-8094-e921266e1f18","Type":"ContainerStarted","Data":"3cc6d198751b8e24e97a7b3f738f125c37777e2940a65a736017b9b53d4f36af"}
Sep 04 22:45:34.564421 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.564385    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-32.ec2.internal" podStartSLOduration=20.564373477 podStartE2EDuration="20.564373477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:17.525119471 +0000 UTC m=+5.635240061" watchObservedRunningTime="2026-09-04 22:45:34.564373477 +0000 UTC m=+22.674494061"
Sep 04 22:45:34.564697 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.564621    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k7nf5" podStartSLOduration=2.9816064669999998 podStartE2EDuration="21.564614412s" totalImagesPullingTime="18.583007945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:34.564186029 +0000 UTC m=+22.674306615" watchObservedRunningTime="2026-09-04 22:45:34.564614412 +0000 UTC m=+22.674734998"
Sep 04 22:45:34.597005 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.596967    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4qbrc" podStartSLOduration=3.012075221 podStartE2EDuration="21.596952817s" totalImagesPullingTime="18.584877596s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:34.596388792 +0000 UTC m=+22.706509381" watchObservedRunningTime="2026-09-04 22:45:34.596952817 +0000 UTC m=+22.707073402"
Sep 04 22:45:34.618434 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.618363    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6748j" podStartSLOduration=3.048047939 podStartE2EDuration="21.618348622s" totalImagesPullingTime="18.570300683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:34.617593681 +0000 UTC m=+22.727714266" watchObservedRunningTime="2026-09-04 22:45:34.618348622 +0000 UTC m=+22.728469208"
Sep 04 22:45:34.640199 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.640160    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qdvpv" podStartSLOduration=2.991686935 podStartE2EDuration="21.640146351s" totalImagesPullingTime="18.648459416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:34.6399564 +0000 UTC m=+22.750076988" watchObservedRunningTime="2026-09-04 22:45:34.640146351 +0000 UTC m=+22.750266937"
Sep 04 22:45:34.658075 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:34.658009    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-57gxp" podStartSLOduration=3.079433851 podStartE2EDuration="21.657989318s" totalImagesPullingTime="18.578555467s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:34.65771407 +0000 UTC m=+22.767834659" watchObservedRunningTime="2026-09-04 22:45:34.657989318 +0000 UTC m=+22.768109908"
Sep 04 22:45:35.414254 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:35.413936    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:35.414766 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:35.414282    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:35.569191 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:35.569151    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n" event={"ID":"6e5ea01c-a8df-4300-b94b-d4b9b0828846","Type":"ContainerStarted","Data":"e9d291ee51bf896bac3121210880a4095689f5de321754c9ede4c8cdab687992"}
Sep 04 22:45:35.569406 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:35.569244    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n" event={"ID":"6e5ea01c-a8df-4300-b94b-d4b9b0828846","Type":"ContainerStarted","Data":"185203964e685152caa5087c5a00a658526cb64c59038840f94759602061bc5e"}
Sep 04 22:45:35.570923 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:35.570887    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l4sc8" event={"ID":"e59abefb-388f-4bd6-a8a6-7db9e73010c2","Type":"ContainerStarted","Data":"05e4d06dc91b4c14300e81204d654db90d705fce165c8f9881cdd93eb281bc0a"}
Sep 04 22:45:35.586738 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:35.586689    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-l4sc8" podStartSLOduration=3.993322988 podStartE2EDuration="22.58666878s" totalImagesPullingTime="18.593345792s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:35.585712331 +0000 UTC m=+23.695832919" watchObservedRunningTime="2026-09-04 22:45:35.58666878 +0000 UTC m=+23.696789396"
Sep 04 22:45:36.098630 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:36.098497    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 04 22:45:36.368866 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:36.368720    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-04T22:45:36.098518059Z","UUID":"adef1ae6-0312-4654-a065-abf175f93cb9","Handler":null,"Name":"","Endpoint":""}
Sep 04 22:45:36.370309 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:36.370282    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 04 22:45:36.370430 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:36.370319    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 04 22:45:36.413837 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:36.413808    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:36.414009 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:36.413915    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:36.414009 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:36.413972    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:36.414125 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:36.414068    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:36.577125 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:36.577083    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9" event={"ID":"a6dfb16b-b631-4bce-8d9f-4d307e58dff2","Type":"ContainerStarted","Data":"97e5a10b8810be71e8d7a79c0a9d046c2450408df8e56c29b9536f972dd9557f"}
Sep 04 22:45:37.413899 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:37.413867    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:37.414051 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:37.413975    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:37.583317 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:37.583277    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n" event={"ID":"6e5ea01c-a8df-4300-b94b-d4b9b0828846","Type":"ContainerStarted","Data":"5876c15dd2cdf4492a24b0ebcfc9344b5e9bd5a7ed1c3532f4332a60d81f0fb6"}
Sep 04 22:45:38.413362 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:38.413328    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:38.413540 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:38.413476    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:38.413540 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:38.413510    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:38.413694 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:38.413623    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:38.841312 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:38.841287    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6748j"
Sep 04 22:45:38.841909 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:38.841892    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6748j"
Sep 04 22:45:39.413904 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.413681    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:39.414030 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:39.413957    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:39.590056 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.590025    2578 generic.go:356] "Generic (PLEG): container finished" podID="aa4adc0d-b12a-4f86-addd-5c39a9fc1255" containerID="93b09c22c7533a099559af30aaa03214f25942485cd043fd44288b5da2a9df84" exitCode=0
Sep 04 22:45:39.590241 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.590113    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9vmrt" event={"ID":"aa4adc0d-b12a-4f86-addd-5c39a9fc1255","Type":"ContainerDied","Data":"93b09c22c7533a099559af30aaa03214f25942485cd043fd44288b5da2a9df84"}
Sep 04 22:45:39.594554 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.594528    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9" event={"ID":"a6dfb16b-b631-4bce-8d9f-4d307e58dff2","Type":"ContainerStarted","Data":"44a9c2fd96ded38c45b16119e85570758a044d77ec0fc90434ab513d7be346e2"}
Sep 04 22:45:39.598796 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.598774    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n" event={"ID":"6e5ea01c-a8df-4300-b94b-d4b9b0828846","Type":"ContainerStarted","Data":"b671345ec8608bae4e910984286b0fc5ae8733a75b31901f13ee5774386b117b"}
Sep 04 22:45:39.599234 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.599211    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6748j"
Sep 04 22:45:39.599423 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.599290    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:39.599423 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.599336    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:39.599423 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.599405    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:39.599889 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.599873    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6748j"
Sep 04 22:45:39.616566 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.616544    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:39.617750 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.617725    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:45:39.649995 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.649952    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n" podStartSLOduration=8.022452362 podStartE2EDuration="26.649938558s" totalImagesPullingTime="18.627486196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:39.648694366 +0000 UTC m=+27.758814952" watchObservedRunningTime="2026-09-04 22:45:39.649938558 +0000 UTC m=+27.760059144"
Sep 04 22:45:39.666697 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:39.666612    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zzxd9" podStartSLOduration=3.561400418 podStartE2EDuration="26.666599818s" totalImagesPullingTime="23.1051994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:39.666409724 +0000 UTC m=+27.776530311" watchObservedRunningTime="2026-09-04 22:45:39.666599818 +0000 UTC m=+27.776720434"
Sep 04 22:45:40.413678 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:40.413628    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:40.414298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:40.413696    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:40.414298 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:40.413786    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:40.414298 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:40.413911    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:40.604098 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:40.603870    2578 generic.go:356] "Generic (PLEG): container finished" podID="aa4adc0d-b12a-4f86-addd-5c39a9fc1255" containerID="dbe1370c77fd0b5e99c9d8e2407f17b6fd195a26e582edfb5d3fe5658c929b68" exitCode=0
Sep 04 22:45:40.604255 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:40.603951    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9vmrt" event={"ID":"aa4adc0d-b12a-4f86-addd-5c39a9fc1255","Type":"ContainerDied","Data":"dbe1370c77fd0b5e99c9d8e2407f17b6fd195a26e582edfb5d3fe5658c929b68"}
Sep 04 22:45:40.995443 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:40.995410    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-77nr4"]
Sep 04 22:45:40.995602 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:40.995546    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:40.995698 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:40.995672    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:40.998687 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:40.998660    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t4vfv"]
Sep 04 22:45:40.998801 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:40.998775    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:40.998938 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:40.998910    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:40.999523 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:40.999497    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9dqmt"]
Sep 04 22:45:40.999650 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:40.999592    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:40.999742 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:40.999716    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:41.608603 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:41.608572    2578 generic.go:356] "Generic (PLEG): container finished" podID="aa4adc0d-b12a-4f86-addd-5c39a9fc1255" containerID="590c48ad7d7e4532a0a2c172130484aabd4f67e70c01b0b715fe15fd3af29607" exitCode=0
Sep 04 22:45:41.609143 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:41.608671    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9vmrt" event={"ID":"aa4adc0d-b12a-4f86-addd-5c39a9fc1255","Type":"ContainerDied","Data":"590c48ad7d7e4532a0a2c172130484aabd4f67e70c01b0b715fe15fd3af29607"}
Sep 04 22:45:42.414174 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:42.414134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:42.414360 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:42.414241    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:42.414360 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:42.414287    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:42.414360 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:42.414345    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:42.730235 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:42.730150    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:42.814347 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:42.814314    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:43.413162 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:43.413131    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:43.413313 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:43.413264    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:43.728172 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:43.728145    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:43.830042 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:43.830008    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:43.930845 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:43.930811    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:44.413630 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:44.413598    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:44.413819 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:44.413722    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:44.413819 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:44.413775    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:44.413947 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:44.413867    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:45.397573 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:45.397362    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:45.413300 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:45.413274    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:45.413455 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:45.413393    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4vfv" podUID="075c9d8c-8c3d-4509-aca6-fa687f5eb1bb"
Sep 04 22:45:46.156324 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:46.156292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:46.156324 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:46.156330    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwprb\" (UniqueName: \"kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb\") pod \"network-check-target-77nr4\" (UID: \"9c9d02b6-4018-4f24-9900-df3d1eadbc59\") " pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:46.156537 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:46.156355    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:46.156537 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:46.156437    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:46.156537 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:46.156440    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:46.156537 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:46.156453    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:46.156537 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:46.156475    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:46.156537 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:46.156484    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rwprb for pod openshift-network-diagnostics/network-check-target-77nr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:46.156537 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:46.156492    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs podName:a7ec67b5-ff79-49d1-b063-8cd7c4a56c84 nodeName:}" failed. No retries permitted until 2026-09-04 22:46:18.156475595 +0000 UTC m=+66.266596162 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs") pod "network-metrics-daemon-9dqmt" (UID: "a7ec67b5-ff79-49d1-b063-8cd7c4a56c84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:46.156537 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:46.156504    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret podName:075c9d8c-8c3d-4509-aca6-fa687f5eb1bb nodeName:}" failed. No retries permitted until 2026-09-04 22:46:18.15649872 +0000 UTC m=+66.266619284 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret") pod "global-pull-secret-syncer-t4vfv" (UID: "075c9d8c-8c3d-4509-aca6-fa687f5eb1bb") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:46.156537 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:46.156520    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb podName:9c9d02b6-4018-4f24-9900-df3d1eadbc59 nodeName:}" failed. No retries permitted until 2026-09-04 22:46:18.156509012 +0000 UTC m=+66.266629593 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwprb" (UniqueName: "kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb") pod "network-check-target-77nr4" (UID: "9c9d02b6-4018-4f24-9900-df3d1eadbc59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:46.413551 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:46.413495    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:46.413551 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:46.413535    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:46.413940 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:46.413622    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:45:46.413940 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:46.413721    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77nr4" podUID="9c9d02b6-4018-4f24-9900-df3d1eadbc59"
Sep 04 22:45:46.624429 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:46.624401    2578 generic.go:356] "Generic (PLEG): container finished" podID="aa4adc0d-b12a-4f86-addd-5c39a9fc1255" containerID="d7615b2f97183e5076491562f7d29fb20e1fa41ec21df9d81c8a24b8eea972ff" exitCode=0
Sep 04 22:45:46.624585 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:46.624455    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9vmrt" event={"ID":"aa4adc0d-b12a-4f86-addd-5c39a9fc1255","Type":"ContainerDied","Data":"d7615b2f97183e5076491562f7d29fb20e1fa41ec21df9d81c8a24b8eea972ff"}
Sep 04 22:45:47.221383 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.221298    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-32.ec2.internal" event="NodeReady"
Sep 04 22:45:47.221516 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.221407    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 04 22:45:47.266028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.265999    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mpx5p"]
Sep 04 22:45:47.279496 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.279469    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hx25v"]
Sep 04 22:45:47.279654 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.279562    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.282343 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.282317    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 04 22:45:47.282839 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.282608    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cv6t6\""
Sep 04 22:45:47.282839 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.282632    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 04 22:45:47.293388 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.293368    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mpx5p"]
Sep 04 22:45:47.293465 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.293399    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hx25v"]
Sep 04 22:45:47.293500 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.293473    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hx25v"
Sep 04 22:45:47.296515 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.296495    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 04 22:45:47.296686 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.296549    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 04 22:45:47.296758 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.296583    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w24qj\""
Sep 04 22:45:47.296758 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.296725    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 04 22:45:47.366132 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.366095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.366293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.366151    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/565701c0-684e-4fc0-9bc5-2ebce20c2100-cert\") pod \"ingress-canary-hx25v\" (UID: \"565701c0-684e-4fc0-9bc5-2ebce20c2100\") " pod="openshift-ingress-canary/ingress-canary-hx25v"
Sep 04 22:45:47.366293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.366170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8242b073-28b9-46e8-8783-9aa8871bcf2e-config-volume\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.366293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.366230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8242b073-28b9-46e8-8783-9aa8871bcf2e-tmp-dir\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.366293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.366269    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2trf9\" (UniqueName: \"kubernetes.io/projected/565701c0-684e-4fc0-9bc5-2ebce20c2100-kube-api-access-2trf9\") pod \"ingress-canary-hx25v\" (UID: \"565701c0-684e-4fc0-9bc5-2ebce20c2100\") " pod="openshift-ingress-canary/ingress-canary-hx25v"
Sep 04 22:45:47.366293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.366290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzzdk\" (UniqueName: \"kubernetes.io/projected/8242b073-28b9-46e8-8783-9aa8871bcf2e-kube-api-access-lzzdk\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.387243 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.387212    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:47.413334 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.413311    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:45:47.416220 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.416195    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 04 22:45:47.441183 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.441156    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:47.466869 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.466849    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8242b073-28b9-46e8-8783-9aa8871bcf2e-config-volume\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.466948 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.466877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8242b073-28b9-46e8-8783-9aa8871bcf2e-tmp-dir\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.466948 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.466897    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2trf9\" (UniqueName: \"kubernetes.io/projected/565701c0-684e-4fc0-9bc5-2ebce20c2100-kube-api-access-2trf9\") pod \"ingress-canary-hx25v\" (UID: \"565701c0-684e-4fc0-9bc5-2ebce20c2100\") " pod="openshift-ingress-canary/ingress-canary-hx25v"
Sep 04 22:45:47.466948 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.466914    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzzdk\" (UniqueName: \"kubernetes.io/projected/8242b073-28b9-46e8-8783-9aa8871bcf2e-kube-api-access-lzzdk\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.467080 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.466953    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.467080 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.466995    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/565701c0-684e-4fc0-9bc5-2ebce20c2100-cert\") pod \"ingress-canary-hx25v\" (UID: \"565701c0-684e-4fc0-9bc5-2ebce20c2100\") " pod="openshift-ingress-canary/ingress-canary-hx25v"
Sep 04 22:45:47.467156 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:47.467138    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:45:47.467219 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:47.467201    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls podName:8242b073-28b9-46e8-8783-9aa8871bcf2e nodeName:}" failed. No retries permitted until 2026-09-04 22:45:47.967180845 +0000 UTC m=+36.077301418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls") pod "dns-default-mpx5p" (UID: "8242b073-28b9-46e8-8783-9aa8871bcf2e") : secret "dns-default-metrics-tls" not found
Sep 04 22:45:47.467308 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.467293    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8242b073-28b9-46e8-8783-9aa8871bcf2e-tmp-dir\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.467455 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.467438    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8242b073-28b9-46e8-8783-9aa8871bcf2e-config-volume\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.470896 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.470873    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/565701c0-684e-4fc0-9bc5-2ebce20c2100-cert\") pod \"ingress-canary-hx25v\" (UID: \"565701c0-684e-4fc0-9bc5-2ebce20c2100\") " pod="openshift-ingress-canary/ingress-canary-hx25v"
Sep 04 22:45:47.475875 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.475821    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzzdk\" (UniqueName: \"kubernetes.io/projected/8242b073-28b9-46e8-8783-9aa8871bcf2e-kube-api-access-lzzdk\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.476049 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.476028    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2trf9\" (UniqueName: \"kubernetes.io/projected/565701c0-684e-4fc0-9bc5-2ebce20c2100-kube-api-access-2trf9\") pod \"ingress-canary-hx25v\" (UID: \"565701c0-684e-4fc0-9bc5-2ebce20c2100\") " pod="openshift-ingress-canary/ingress-canary-hx25v"
Sep 04 22:45:47.602282 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.602246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hx25v"
Sep 04 22:45:47.630691 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.630658    2578 generic.go:356] "Generic (PLEG): container finished" podID="aa4adc0d-b12a-4f86-addd-5c39a9fc1255" containerID="1774ca11be1717712a71351f614c19be591b3eb9c4c4123bf2a25e4904ba4ada" exitCode=0
Sep 04 22:45:47.630804 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.630730    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9vmrt" event={"ID":"aa4adc0d-b12a-4f86-addd-5c39a9fc1255","Type":"ContainerDied","Data":"1774ca11be1717712a71351f614c19be591b3eb9c4c4123bf2a25e4904ba4ada"}
Sep 04 22:45:47.921353 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.921324    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hx25v"]
Sep 04 22:45:47.924278 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:45:47.924254    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod565701c0_684e_4fc0_9bc5_2ebce20c2100.slice/crio-5a824ad344b49c924a5587c71f18dbe8b8ca920013869dc5ebd8ea9089570f9b WatchSource:0}: Error finding container 5a824ad344b49c924a5587c71f18dbe8b8ca920013869dc5ebd8ea9089570f9b: Status 404 returned error can't find the container with id 5a824ad344b49c924a5587c71f18dbe8b8ca920013869dc5ebd8ea9089570f9b
Sep 04 22:45:47.971217 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:47.971188    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:47.971367 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:47.971303    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:45:47.971367 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:47.971354    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls podName:8242b073-28b9-46e8-8783-9aa8871bcf2e nodeName:}" failed. No retries permitted until 2026-09-04 22:45:48.971340524 +0000 UTC m=+37.081461087 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls") pod "dns-default-mpx5p" (UID: "8242b073-28b9-46e8-8783-9aa8871bcf2e") : secret "dns-default-metrics-tls" not found
Sep 04 22:45:48.413175 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.413142    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:45:48.413326 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.413142    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:45:48.416076 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.416052    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 04 22:45:48.416847 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.416828    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sxf76\""
Sep 04 22:45:48.417428 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.416901    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 04 22:45:48.417428 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.416926    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gnghc\""
Sep 04 22:45:48.417428 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.416934    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 04 22:45:48.637313 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.637243    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9vmrt" event={"ID":"aa4adc0d-b12a-4f86-addd-5c39a9fc1255","Type":"ContainerStarted","Data":"a51213a3b0da442c24b57bee31134f0c29741a891d522482b6b357a10b40ce9c"}
Sep 04 22:45:48.638530 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.638500    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hx25v" event={"ID":"565701c0-684e-4fc0-9bc5-2ebce20c2100","Type":"ContainerStarted","Data":"5a824ad344b49c924a5587c71f18dbe8b8ca920013869dc5ebd8ea9089570f9b"}
Sep 04 22:45:48.661930 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.661876    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9vmrt" podStartSLOduration=6.45951045 podStartE2EDuration="35.661856423s" totalImagesPullingTime="29.202345973s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:48.66094972 +0000 UTC m=+36.771070310" watchObservedRunningTime="2026-09-04 22:45:48.661856423 +0000 UTC m=+36.771977016"
Sep 04 22:45:48.785115 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.785033    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:48.840055 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.840019    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:48.979961 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:48.979921    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:48.980130 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:48.980087    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:45:48.980208 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:48.980164    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls podName:8242b073-28b9-46e8-8783-9aa8871bcf2e nodeName:}" failed. No retries permitted until 2026-09-04 22:45:50.980142736 +0000 UTC m=+39.090263320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls") pod "dns-default-mpx5p" (UID: "8242b073-28b9-46e8-8783-9aa8871bcf2e") : secret "dns-default-metrics-tls" not found
Sep 04 22:45:50.645564 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:50.645492    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hx25v" event={"ID":"565701c0-684e-4fc0-9bc5-2ebce20c2100","Type":"ContainerStarted","Data":"029dd405b779522058cb2430ca4b8d7b6c2dc13990a8add06c515f392277cdbf"}
Sep 04 22:45:50.664915 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:50.664808    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hx25v" podStartSLOduration=1.181836302 podStartE2EDuration="3.66478814s" totalImagesPullingTime="2.482951838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:45:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:50.66378707 +0000 UTC m=+38.773907655" watchObservedRunningTime="2026-09-04 22:45:50.66478814 +0000 UTC m=+38.774908731"
Sep 04 22:45:50.995556 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:50.995519    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:50.995724 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:50.995704    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:45:50.995799 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:50.995786    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls podName:8242b073-28b9-46e8-8783-9aa8871bcf2e nodeName:}" failed. No retries permitted until 2026-09-04 22:45:54.99576506 +0000 UTC m=+43.105885638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls") pod "dns-default-mpx5p" (UID: "8242b073-28b9-46e8-8783-9aa8871bcf2e") : secret "dns-default-metrics-tls" not found
Sep 04 22:45:52.442129 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:52.442098    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:52.492659 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:52.492609    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:53.786829 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:53.786793    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:53.836312 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:53.836275    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:55.028720 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:55.028687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:45:55.029084 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:55.028805    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:45:55.029084 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:45:55.028857    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls podName:8242b073-28b9-46e8-8783-9aa8871bcf2e nodeName:}" failed. No retries permitted until 2026-09-04 22:46:03.028843585 +0000 UTC m=+51.138964149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls") pod "dns-default-mpx5p" (UID: "8242b073-28b9-46e8-8783-9aa8871bcf2e") : secret "dns-default-metrics-tls" not found
Sep 04 22:45:55.066600 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:55.066566    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:45:55.124041 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:45:55.124014    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:46:03.081886 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:03.081842    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:46:03.082352 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:46:03.082011    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:46:03.082352 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:46:03.082080    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls podName:8242b073-28b9-46e8-8783-9aa8871bcf2e nodeName:}" failed. No retries permitted until 2026-09-04 22:46:19.082064975 +0000 UTC m=+67.192185540 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls") pod "dns-default-mpx5p" (UID: "8242b073-28b9-46e8-8783-9aa8871bcf2e") : secret "dns-default-metrics-tls" not found
Sep 04 22:46:06.755534 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:06.755493    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:46:06.849283 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:06.849249    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:46:11.619187 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:11.619154    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-znc4n"
Sep 04 22:46:11.768401 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:11.768365    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:46:11.828658 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:11.828613    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:46:15.730489 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:15.730453    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:46:15.831819 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:15.831785    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:46:15.883046 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:15.883012    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:46:18.188804 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.188757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:46:18.189194 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.188831    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:46:18.189194 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.188879    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwprb\" (UniqueName: \"kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb\") pod \"network-check-target-77nr4\" (UID: \"9c9d02b6-4018-4f24-9900-df3d1eadbc59\") " pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:46:18.191938 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.191916    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 04 22:46:18.191938 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.191930    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 04 22:46:18.192024 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.191937    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 04 22:46:18.198971 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:46:18.198954    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 04 22:46:18.199075 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:46:18.199008    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs podName:a7ec67b5-ff79-49d1-b063-8cd7c4a56c84 nodeName:}" failed. No retries permitted until 2026-09-04 22:47:22.198991906 +0000 UTC m=+130.309112470 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs") pod "network-metrics-daemon-9dqmt" (UID: "a7ec67b5-ff79-49d1-b063-8cd7c4a56c84") : secret "metrics-daemon-secret" not found
Sep 04 22:46:18.200987 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.200966    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 04 22:46:18.202438 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.202411    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/075c9d8c-8c3d-4509-aca6-fa687f5eb1bb-original-pull-secret\") pod \"global-pull-secret-syncer-t4vfv\" (UID: \"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb\") " pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:46:18.211669 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.211630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwprb\" (UniqueName: \"kubernetes.io/projected/9c9d02b6-4018-4f24-9900-df3d1eadbc59-kube-api-access-rwprb\") pod \"network-check-target-77nr4\" (UID: \"9c9d02b6-4018-4f24-9900-df3d1eadbc59\") " pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:46:18.321205 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.321165    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4vfv"
Sep 04 22:46:18.424934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.424910    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gnghc\""
Sep 04 22:46:18.432760 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.432735    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:46:18.588906 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.588876    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t4vfv"]
Sep 04 22:46:18.591855 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:46:18.591833    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod075c9d8c_8c3d_4509_aca6_fa687f5eb1bb.slice/crio-d5c0b802193c984fc188c11bca9e0dd018a66843277fd35d02f8229244d5b83f WatchSource:0}: Error finding container d5c0b802193c984fc188c11bca9e0dd018a66843277fd35d02f8229244d5b83f: Status 404 returned error can't find the container with id d5c0b802193c984fc188c11bca9e0dd018a66843277fd35d02f8229244d5b83f
Sep 04 22:46:18.696198 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.696123    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-77nr4"]
Sep 04 22:46:18.699446 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:46:18.699415    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c9d02b6_4018_4f24_9900_df3d1eadbc59.slice/crio-21441f267d9040d0d258548f7855ae211c960b9a46ae4a26e444c7e9ee7a52ba WatchSource:0}: Error finding container 21441f267d9040d0d258548f7855ae211c960b9a46ae4a26e444c7e9ee7a52ba: Status 404 returned error can't find the container with id 21441f267d9040d0d258548f7855ae211c960b9a46ae4a26e444c7e9ee7a52ba
Sep 04 22:46:18.709760 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.709735    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-77nr4" event={"ID":"9c9d02b6-4018-4f24-9900-df3d1eadbc59","Type":"ContainerStarted","Data":"21441f267d9040d0d258548f7855ae211c960b9a46ae4a26e444c7e9ee7a52ba"}
Sep 04 22:46:18.710686 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:18.710665    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t4vfv" event={"ID":"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb","Type":"ContainerStarted","Data":"d5c0b802193c984fc188c11bca9e0dd018a66843277fd35d02f8229244d5b83f"}
Sep 04 22:46:19.094770 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:19.094739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:46:19.094912 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:46:19.094888    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:46:19.094948 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:46:19.094942    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls podName:8242b073-28b9-46e8-8783-9aa8871bcf2e nodeName:}" failed. No retries permitted until 2026-09-04 22:46:51.094927194 +0000 UTC m=+99.205047762 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls") pod "dns-default-mpx5p" (UID: "8242b073-28b9-46e8-8783-9aa8871bcf2e") : secret "dns-default-metrics-tls" not found
Sep 04 22:46:19.929251 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:19.929181    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:46:19.991873 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:19.991835    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:46:23.724999 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:23.724958    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t4vfv" event={"ID":"075c9d8c-8c3d-4509-aca6-fa687f5eb1bb","Type":"ContainerStarted","Data":"31ae933784054502c69cacd8d8bdedc0f715979bb22def7559c2bd586308577a"}
Sep 04 22:46:23.726415 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:23.726362    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-77nr4" event={"ID":"9c9d02b6-4018-4f24-9900-df3d1eadbc59","Type":"ContainerStarted","Data":"4fc291ed72d3bb7ddaa73995ae219bcddb3a1cf54aadd38cf92d026e78d32720"}
Sep 04 22:46:23.726518 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:23.726449    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:46:23.740168 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:23.740115    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t4vfv" podStartSLOduration=66.212995649 podStartE2EDuration="1m10.740097864s" totalImagesPullingTime="4.527102215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:46:23.739629635 +0000 UTC m=+71.849750221" watchObservedRunningTime="2026-09-04 22:46:23.740097864 +0000 UTC m=+71.850218452"
Sep 04 22:46:23.756625 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:23.756583    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-77nr4" podStartSLOduration=66.339808991 podStartE2EDuration="1m10.756569968s" totalImagesPullingTime="4.416760977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:46:23.755911066 +0000 UTC m=+71.866031653" watchObservedRunningTime="2026-09-04 22:46:23.756569968 +0000 UTC m=+71.866690554"
Sep 04 22:46:51.101749 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:51.101694    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:46:51.102177 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:46:51.101813    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:46:51.102177 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:46:51.101875    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls podName:8242b073-28b9-46e8-8783-9aa8871bcf2e nodeName:}" failed. No retries permitted until 2026-09-04 22:47:55.101860492 +0000 UTC m=+163.211981056 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls") pod "dns-default-mpx5p" (UID: "8242b073-28b9-46e8-8783-9aa8871bcf2e") : secret "dns-default-metrics-tls" not found
Sep 04 22:46:51.413850 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:51.413768    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:46:54.732089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:46:54.732057    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-77nr4"
Sep 04 22:47:02.065503 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.065471    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w"]
Sep 04 22:47:02.068357 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.068341    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w"
Sep 04 22:47:02.070967 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.070946    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 04 22:47:02.071102 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.070949    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 04 22:47:02.071989 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.071725    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7d62x\""
Sep 04 22:47:02.074953 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.074907    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 04 22:47:02.075142 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.075102    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 04 22:47:02.079540 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.079519    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w"]
Sep 04 22:47:02.176384 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.176349    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34fe39ee-e71f-4e93-98d5-c75a3c68fa2b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w\" (UID: \"34fe39ee-e71f-4e93-98d5-c75a3c68fa2b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w"
Sep 04 22:47:02.176384 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.176394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdjn7\" (UniqueName: \"kubernetes.io/projected/34fe39ee-e71f-4e93-98d5-c75a3c68fa2b-kube-api-access-tdjn7\") pod \"managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w\" (UID: \"34fe39ee-e71f-4e93-98d5-c75a3c68fa2b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w"
Sep 04 22:47:02.277257 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.277215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34fe39ee-e71f-4e93-98d5-c75a3c68fa2b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w\" (UID: \"34fe39ee-e71f-4e93-98d5-c75a3c68fa2b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w"
Sep 04 22:47:02.277353 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.277266    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdjn7\" (UniqueName: \"kubernetes.io/projected/34fe39ee-e71f-4e93-98d5-c75a3c68fa2b-kube-api-access-tdjn7\") pod \"managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w\" (UID: \"34fe39ee-e71f-4e93-98d5-c75a3c68fa2b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w"
Sep 04 22:47:02.279856 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.279824    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34fe39ee-e71f-4e93-98d5-c75a3c68fa2b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w\" (UID: \"34fe39ee-e71f-4e93-98d5-c75a3c68fa2b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w"
Sep 04 22:47:02.285767 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.285743    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdjn7\" (UniqueName: \"kubernetes.io/projected/34fe39ee-e71f-4e93-98d5-c75a3c68fa2b-kube-api-access-tdjn7\") pod \"managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w\" (UID: \"34fe39ee-e71f-4e93-98d5-c75a3c68fa2b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w"
Sep 04 22:47:02.387274 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.387250    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w"
Sep 04 22:47:02.653750 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.653670    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w"]
Sep 04 22:47:02.657079 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:47:02.657053    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34fe39ee_e71f_4e93_98d5_c75a3c68fa2b.slice/crio-b8933e76fe589973bd262578f00d87ed34f956ed74c88117aeae0b0d1a2c9a23 WatchSource:0}: Error finding container b8933e76fe589973bd262578f00d87ed34f956ed74c88117aeae0b0d1a2c9a23: Status 404 returned error can't find the container with id b8933e76fe589973bd262578f00d87ed34f956ed74c88117aeae0b0d1a2c9a23
Sep 04 22:47:02.822947 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:02.822911    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w" event={"ID":"34fe39ee-e71f-4e93-98d5-c75a3c68fa2b","Type":"ContainerStarted","Data":"b8933e76fe589973bd262578f00d87ed34f956ed74c88117aeae0b0d1a2c9a23"}
Sep 04 22:47:05.831574 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:05.831539    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w" event={"ID":"34fe39ee-e71f-4e93-98d5-c75a3c68fa2b","Type":"ContainerStarted","Data":"1f4218b8fdd3f84c01b2ee5d6f305466625b01f90bc984ca4d486c49b1c262b7"}
Sep 04 22:47:05.846966 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:05.846913    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w" podStartSLOduration=1.585169101 podStartE2EDuration="3.846897948s" totalImagesPullingTime="2.261728847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:47:05.846335542 +0000 UTC m=+113.956456127" watchObservedRunningTime="2026-09-04 22:47:05.846897948 +0000 UTC m=+113.957018533"
Sep 04 22:47:22.213510 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:22.213451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:47:22.214055 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:47:22.213629    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 04 22:47:22.214055 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:47:22.213763    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs podName:a7ec67b5-ff79-49d1-b063-8cd7c4a56c84 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:24.213740059 +0000 UTC m=+252.323860642 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs") pod "network-metrics-daemon-9dqmt" (UID: "a7ec67b5-ff79-49d1-b063-8cd7c4a56c84") : secret "metrics-daemon-secret" not found
Sep 04 22:47:50.289559 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:47:50.289511    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-mpx5p" podUID="8242b073-28b9-46e8-8783-9aa8871bcf2e"
Sep 04 22:47:50.937385 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:50.937355    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:47:51.427538 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:47:51.427502    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9dqmt" podUID="a7ec67b5-ff79-49d1-b063-8cd7c4a56c84"
Sep 04 22:47:55.132042 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:47:55.132001    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:47:55.132433 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:47:55.132166    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:47:55.132433 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:47:55.132243    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls podName:8242b073-28b9-46e8-8783-9aa8871bcf2e nodeName:}" failed. No retries permitted until 2026-09-04 22:49:57.132225977 +0000 UTC m=+285.242346547 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls") pod "dns-default-mpx5p" (UID: "8242b073-28b9-46e8-8783-9aa8871bcf2e") : secret "dns-default-metrics-tls" not found
Sep 04 22:48:06.413613 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:06.413523    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:48:24.488565 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:24.488530    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:48:54.128080 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.128049    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-ckz9h"]
Sep 04 22:48:54.130749 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.130727    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:48:54.132390 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.132197    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"]
Sep 04 22:48:54.134808 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.134788    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-58vxq"]
Sep 04 22:48:54.134938 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.134922    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"
Sep 04 22:48:54.136699 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.136677    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-m2w6p\""
Sep 04 22:48:54.136801 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.136699    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 04 22:48:54.136801 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.136713    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 04 22:48:54.137317 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.137298    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-67c5f4dcb-cnpgt"]
Sep 04 22:48:54.137434 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.137393    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.137509 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.137471    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 04 22:48:54.137589 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.137476    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 04 22:48:54.138026 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.137966    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 04 22:48:54.138026 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.138009    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5h6qj\""
Sep 04 22:48:54.138154 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.138051    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 04 22:48:54.139847 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.139832    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77656967dd-bw4qw"]
Sep 04 22:48:54.139991 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.139977    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.140138 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.140121    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 04 22:48:54.140261 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.140138    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-f9xxx\""
Sep 04 22:48:54.142169 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.142148    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 04 22:48:54.142409 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.142386    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 04 22:48:54.142506 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.142494    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 04 22:48:54.142576 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.142552    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.142772 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.142751    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 04 22:48:54.142859 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.142774    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 04 22:48:54.143135 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.143117    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 04 22:48:54.143213 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.143122    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 04 22:48:54.143427 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.143412    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 04 22:48:54.143730 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.143711    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 04 22:48:54.143924 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.143909    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f7cw8\""
Sep 04 22:48:54.146007 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.145987    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 04 22:48:54.146104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.145987    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 04 22:48:54.146186 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.146169    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 04 22:48:54.147062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.147042    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6zcvv\""
Sep 04 22:48:54.149256 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.149233    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"]
Sep 04 22:48:54.149883 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.149859    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 04 22:48:54.150127 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.150107    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-ckz9h"]
Sep 04 22:48:54.151175 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.151126    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-58vxq"]
Sep 04 22:48:54.153465 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.153439    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 04 22:48:54.154362 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.154266    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77656967dd-bw4qw"]
Sep 04 22:48:54.167325 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.167303    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-67c5f4dcb-cnpgt"]
Sep 04 22:48:54.221864 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.221836    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b1acb3a-ce94-4fa2-963b-f19969e1f344-config\") pod \"service-ca-operator-859f94c4b-9sjbp\" (UID: \"0b1acb3a-ce94-4fa2-963b-f19969e1f344\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"
Sep 04 22:48:54.221864 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.221867    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c89c14f-e6b5-4754-9182-64a5858d0e2e-config\") pod \"console-operator-85b454874b-58vxq\" (UID: \"6c89c14f-e6b5-4754-9182-64a5858d0e2e\") " pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.222062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.221886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c89c14f-e6b5-4754-9182-64a5858d0e2e-serving-cert\") pod \"console-operator-85b454874b-58vxq\" (UID: \"6c89c14f-e6b5-4754-9182-64a5858d0e2e\") " pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.222062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.221904    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.222062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.221981    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ckz9h\" (UID: \"d5878698-1651-4f18-b257-e5f0fa24e1d0\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:48:54.222062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.222007    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d5878698-1651-4f18-b257-e5f0fa24e1d0-nginx-conf\") pod \"networking-console-plugin-79977f9844-ckz9h\" (UID: \"d5878698-1651-4f18-b257-e5f0fa24e1d0\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:48:54.222062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.222021    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n786p\" (UniqueName: \"kubernetes.io/projected/0b1acb3a-ce94-4fa2-963b-f19969e1f344-kube-api-access-n786p\") pod \"service-ca-operator-859f94c4b-9sjbp\" (UID: \"0b1acb3a-ce94-4fa2-963b-f19969e1f344\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"
Sep 04 22:48:54.222062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.222037    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.222062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.222053    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c89c14f-e6b5-4754-9182-64a5858d0e2e-trusted-ca\") pod \"console-operator-85b454874b-58vxq\" (UID: \"6c89c14f-e6b5-4754-9182-64a5858d0e2e\") " pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.222278 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.222073    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8h6b\" (UniqueName: \"kubernetes.io/projected/6c89c14f-e6b5-4754-9182-64a5858d0e2e-kube-api-access-r8h6b\") pod \"console-operator-85b454874b-58vxq\" (UID: \"6c89c14f-e6b5-4754-9182-64a5858d0e2e\") " pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.222278 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.222100    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-default-certificate\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.222278 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.222141    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b1acb3a-ce94-4fa2-963b-f19969e1f344-serving-cert\") pod \"service-ca-operator-859f94c4b-9sjbp\" (UID: \"0b1acb3a-ce94-4fa2-963b-f19969e1f344\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"
Sep 04 22:48:54.222278 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.222178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-stats-auth\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.222278 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.222213    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pmhh\" (UniqueName: \"kubernetes.io/projected/fbbca652-c23c-4eaf-b908-7d724be0ab91-kube-api-access-4pmhh\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.302655 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.302614    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:48:54.322667 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.322630    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c98bf6bb-910a-4526-b463-7d6761159691-installation-pull-secrets\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.322736 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.322676    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c89c14f-e6b5-4754-9182-64a5858d0e2e-serving-cert\") pod \"console-operator-85b454874b-58vxq\" (UID: \"6c89c14f-e6b5-4754-9182-64a5858d0e2e\") " pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.322774 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.322730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ckz9h\" (UID: \"d5878698-1651-4f18-b257-e5f0fa24e1d0\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:48:54.322774 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.322759    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n786p\" (UniqueName: \"kubernetes.io/projected/0b1acb3a-ce94-4fa2-963b-f19969e1f344-kube-api-access-n786p\") pod \"service-ca-operator-859f94c4b-9sjbp\" (UID: \"0b1acb3a-ce94-4fa2-963b-f19969e1f344\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"
Sep 04 22:48:54.322868 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.322778    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.322868 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.322795    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c98bf6bb-910a-4526-b463-7d6761159691-registry-certificates\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.322868 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.322822    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 04 22:48:54.322980 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.322824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-bound-sa-token\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.322980 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.322881    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 04 22:48:54.322980 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.322889    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert podName:d5878698-1651-4f18-b257-e5f0fa24e1d0 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:54.822870507 +0000 UTC m=+222.932991092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-ckz9h" (UID: "d5878698-1651-4f18-b257-e5f0fa24e1d0") : secret "networking-console-plugin-cert" not found
Sep 04 22:48:54.322980 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.322967    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs podName:fbbca652-c23c-4eaf-b908-7d724be0ab91 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:54.82294916 +0000 UTC m=+222.933069739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs") pod "router-default-67c5f4dcb-cnpgt" (UID: "fbbca652-c23c-4eaf-b908-7d724be0ab91") : secret "router-metrics-certs-default" not found
Sep 04 22:48:54.323117 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323007    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d5878698-1651-4f18-b257-e5f0fa24e1d0-nginx-conf\") pod \"networking-console-plugin-79977f9844-ckz9h\" (UID: \"d5878698-1651-4f18-b257-e5f0fa24e1d0\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:48:54.323117 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323036    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8h6b\" (UniqueName: \"kubernetes.io/projected/6c89c14f-e6b5-4754-9182-64a5858d0e2e-kube-api-access-r8h6b\") pod \"console-operator-85b454874b-58vxq\" (UID: \"6c89c14f-e6b5-4754-9182-64a5858d0e2e\") " pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.323117 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b1acb3a-ce94-4fa2-963b-f19969e1f344-serving-cert\") pod \"service-ca-operator-859f94c4b-9sjbp\" (UID: \"0b1acb3a-ce94-4fa2-963b-f19969e1f344\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"
Sep 04 22:48:54.323117 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323105    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c98bf6bb-910a-4526-b463-7d6761159691-ca-trust-extracted\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.323284 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323135    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjlg5\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-kube-api-access-tjlg5\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.323284 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323162    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c98bf6bb-910a-4526-b463-7d6761159691-image-registry-private-configuration\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.323284 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323184    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c98bf6bb-910a-4526-b463-7d6761159691-trusted-ca\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.323284 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323210    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-stats-auth\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.323284 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323237    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b1acb3a-ce94-4fa2-963b-f19969e1f344-config\") pod \"service-ca-operator-859f94c4b-9sjbp\" (UID: \"0b1acb3a-ce94-4fa2-963b-f19969e1f344\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"
Sep 04 22:48:54.323523 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c89c14f-e6b5-4754-9182-64a5858d0e2e-config\") pod \"console-operator-85b454874b-58vxq\" (UID: \"6c89c14f-e6b5-4754-9182-64a5858d0e2e\") " pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.323523 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323338    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.323523 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323370    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pmhh\" (UniqueName: \"kubernetes.io/projected/fbbca652-c23c-4eaf-b908-7d724be0ab91-kube-api-access-4pmhh\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.323523 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.323523 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323435    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c89c14f-e6b5-4754-9182-64a5858d0e2e-trusted-ca\") pod \"console-operator-85b454874b-58vxq\" (UID: \"6c89c14f-e6b5-4754-9182-64a5858d0e2e\") " pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.323523 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323460    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-default-certificate\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.323523 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.323491    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle podName:fbbca652-c23c-4eaf-b908-7d724be0ab91 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:54.823472307 +0000 UTC m=+222.933592876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle") pod "router-default-67c5f4dcb-cnpgt" (UID: "fbbca652-c23c-4eaf-b908-7d724be0ab91") : configmap references non-existent config key: service-ca.crt
Sep 04 22:48:54.323906 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.323854    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d5878698-1651-4f18-b257-e5f0fa24e1d0-nginx-conf\") pod \"networking-console-plugin-79977f9844-ckz9h\" (UID: \"d5878698-1651-4f18-b257-e5f0fa24e1d0\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:48:54.324193 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.324169    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c89c14f-e6b5-4754-9182-64a5858d0e2e-config\") pod \"console-operator-85b454874b-58vxq\" (UID: \"6c89c14f-e6b5-4754-9182-64a5858d0e2e\") " pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.324314 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.324280    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b1acb3a-ce94-4fa2-963b-f19969e1f344-config\") pod \"service-ca-operator-859f94c4b-9sjbp\" (UID: \"0b1acb3a-ce94-4fa2-963b-f19969e1f344\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"
Sep 04 22:48:54.324519 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.324496    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c89c14f-e6b5-4754-9182-64a5858d0e2e-trusted-ca\") pod \"console-operator-85b454874b-58vxq\" (UID: \"6c89c14f-e6b5-4754-9182-64a5858d0e2e\") " pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.325356 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.325325    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c89c14f-e6b5-4754-9182-64a5858d0e2e-serving-cert\") pod \"console-operator-85b454874b-58vxq\" (UID: \"6c89c14f-e6b5-4754-9182-64a5858d0e2e\") " pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.325748 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.325720    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b1acb3a-ce94-4fa2-963b-f19969e1f344-serving-cert\") pod \"service-ca-operator-859f94c4b-9sjbp\" (UID: \"0b1acb3a-ce94-4fa2-963b-f19969e1f344\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"
Sep 04 22:48:54.326133 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.326115    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-default-certificate\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.326190 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.326166    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-stats-auth\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.331364 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.331337    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8h6b\" (UniqueName: \"kubernetes.io/projected/6c89c14f-e6b5-4754-9182-64a5858d0e2e-kube-api-access-r8h6b\") pod \"console-operator-85b454874b-58vxq\" (UID: \"6c89c14f-e6b5-4754-9182-64a5858d0e2e\") " pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.331881 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.331863    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n786p\" (UniqueName: \"kubernetes.io/projected/0b1acb3a-ce94-4fa2-963b-f19969e1f344-kube-api-access-n786p\") pod \"service-ca-operator-859f94c4b-9sjbp\" (UID: \"0b1acb3a-ce94-4fa2-963b-f19969e1f344\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"
Sep 04 22:48:54.332206 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.332190    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pmhh\" (UniqueName: \"kubernetes.io/projected/fbbca652-c23c-4eaf-b908-7d724be0ab91-kube-api-access-4pmhh\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.401309 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.401240    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:48:54.424289 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.424264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c98bf6bb-910a-4526-b463-7d6761159691-installation-pull-secrets\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.424393 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.424322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c98bf6bb-910a-4526-b463-7d6761159691-registry-certificates\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.424393 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.424342    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-bound-sa-token\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.424393 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.424371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c98bf6bb-910a-4526-b463-7d6761159691-ca-trust-extracted\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.424393 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.424386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjlg5\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-kube-api-access-tjlg5\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.424571 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.424402    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c98bf6bb-910a-4526-b463-7d6761159691-image-registry-private-configuration\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.424571 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.424427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c98bf6bb-910a-4526-b463-7d6761159691-trusted-ca\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.424571 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.424472    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.424739 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.424571    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:48:54.424739 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.424585    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77656967dd-bw4qw: secret "image-registry-tls" not found
Sep 04 22:48:54.424739 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.424665    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls podName:c98bf6bb-910a-4526-b463-7d6761159691 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:54.924629307 +0000 UTC m=+223.034749886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls") pod "image-registry-77656967dd-bw4qw" (UID: "c98bf6bb-910a-4526-b463-7d6761159691") : secret "image-registry-tls" not found
Sep 04 22:48:54.424906 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.424868    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c98bf6bb-910a-4526-b463-7d6761159691-ca-trust-extracted\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.425196 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.425175    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c98bf6bb-910a-4526-b463-7d6761159691-registry-certificates\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.425402 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.425380    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c98bf6bb-910a-4526-b463-7d6761159691-trusted-ca\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.426678 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.426659    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c98bf6bb-910a-4526-b463-7d6761159691-image-registry-private-configuration\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.426856 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.426836    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c98bf6bb-910a-4526-b463-7d6761159691-installation-pull-secrets\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.432725 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.432695    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-bound-sa-token\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.432873 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.432854    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjlg5\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-kube-api-access-tjlg5\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.447854 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.447825    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"
Sep 04 22:48:54.456944 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.456926    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:48:54.718188 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.718114    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp"]
Sep 04 22:48:54.720958 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:48:54.720928    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b1acb3a_ce94_4fa2_963b_f19969e1f344.slice/crio-3af2645841aa1e05b0200fd21958aae6a138dcec0e46a789bad4e9afa8e16cf5 WatchSource:0}: Error finding container 3af2645841aa1e05b0200fd21958aae6a138dcec0e46a789bad4e9afa8e16cf5: Status 404 returned error can't find the container with id 3af2645841aa1e05b0200fd21958aae6a138dcec0e46a789bad4e9afa8e16cf5
Sep 04 22:48:54.737968 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.737819    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-58vxq"]
Sep 04 22:48:54.739576 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:48:54.739553    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c89c14f_e6b5_4754_9182_64a5858d0e2e.slice/crio-4a7b5bc83d354762104fed129184378e1b2f513c75621103798774952beb50da WatchSource:0}: Error finding container 4a7b5bc83d354762104fed129184378e1b2f513c75621103798774952beb50da: Status 404 returned error can't find the container with id 4a7b5bc83d354762104fed129184378e1b2f513c75621103798774952beb50da
Sep 04 22:48:54.826929 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.826893    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ckz9h\" (UID: \"d5878698-1651-4f18-b257-e5f0fa24e1d0\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:48:54.827066 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.826938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.827066 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.826977    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:54.827066 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.827046    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 04 22:48:54.827231 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.827106    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle podName:fbbca652-c23c-4eaf-b908-7d724be0ab91 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:55.827092441 +0000 UTC m=+223.937213005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle") pod "router-default-67c5f4dcb-cnpgt" (UID: "fbbca652-c23c-4eaf-b908-7d724be0ab91") : configmap references non-existent config key: service-ca.crt
Sep 04 22:48:54.827231 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.827111    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 04 22:48:54.827231 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.827124    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert podName:d5878698-1651-4f18-b257-e5f0fa24e1d0 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:55.827118046 +0000 UTC m=+223.937238610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-ckz9h" (UID: "d5878698-1651-4f18-b257-e5f0fa24e1d0") : secret "networking-console-plugin-cert" not found
Sep 04 22:48:54.827231 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.827168    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs podName:fbbca652-c23c-4eaf-b908-7d724be0ab91 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:55.827149982 +0000 UTC m=+223.937270559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs") pod "router-default-67c5f4dcb-cnpgt" (UID: "fbbca652-c23c-4eaf-b908-7d724be0ab91") : secret "router-metrics-certs-default" not found
Sep 04 22:48:54.927707 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:54.927675    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:54.927844 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.927798    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:48:54.927844 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.927810    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77656967dd-bw4qw: secret "image-registry-tls" not found
Sep 04 22:48:54.927937 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:54.927861    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls podName:c98bf6bb-910a-4526-b463-7d6761159691 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:55.927845791 +0000 UTC m=+224.037966354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls") pod "image-registry-77656967dd-bw4qw" (UID: "c98bf6bb-910a-4526-b463-7d6761159691") : secret "image-registry-tls" not found
Sep 04 22:48:55.089120 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:55.089082    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-58vxq" event={"ID":"6c89c14f-e6b5-4754-9182-64a5858d0e2e","Type":"ContainerStarted","Data":"4a7b5bc83d354762104fed129184378e1b2f513c75621103798774952beb50da"}
Sep 04 22:48:55.090102 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:55.090077    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp" event={"ID":"0b1acb3a-ce94-4fa2-963b-f19969e1f344","Type":"ContainerStarted","Data":"3af2645841aa1e05b0200fd21958aae6a138dcec0e46a789bad4e9afa8e16cf5"}
Sep 04 22:48:55.835795 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:55.835730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ckz9h\" (UID: \"d5878698-1651-4f18-b257-e5f0fa24e1d0\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:48:55.836391 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:55.835801    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:55.836391 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:55.835868    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:55.836391 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:55.836070    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle podName:fbbca652-c23c-4eaf-b908-7d724be0ab91 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:57.836050016 +0000 UTC m=+225.946170598 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle") pod "router-default-67c5f4dcb-cnpgt" (UID: "fbbca652-c23c-4eaf-b908-7d724be0ab91") : configmap references non-existent config key: service-ca.crt
Sep 04 22:48:55.836391 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:55.836191    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 04 22:48:55.836391 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:55.836246    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs podName:fbbca652-c23c-4eaf-b908-7d724be0ab91 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:57.836233179 +0000 UTC m=+225.946353748 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs") pod "router-default-67c5f4dcb-cnpgt" (UID: "fbbca652-c23c-4eaf-b908-7d724be0ab91") : secret "router-metrics-certs-default" not found
Sep 04 22:48:55.836391 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:55.836256    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 04 22:48:55.836391 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:55.836325    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert podName:d5878698-1651-4f18-b257-e5f0fa24e1d0 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:57.836306709 +0000 UTC m=+225.946427279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-ckz9h" (UID: "d5878698-1651-4f18-b257-e5f0fa24e1d0") : secret "networking-console-plugin-cert" not found
Sep 04 22:48:55.936429 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:55.936387    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:55.936636 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:55.936535    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:48:55.936636 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:55.936565    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77656967dd-bw4qw: secret "image-registry-tls" not found
Sep 04 22:48:55.936636 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:55.936625    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls podName:c98bf6bb-910a-4526-b463-7d6761159691 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:57.936610805 +0000 UTC m=+226.046731369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls") pod "image-registry-77656967dd-bw4qw" (UID: "c98bf6bb-910a-4526-b463-7d6761159691") : secret "image-registry-tls" not found
Sep 04 22:48:57.179872 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:57.179848    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:48:57.280758 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:57.280717    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:48:57.854065 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:57.854023    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ckz9h\" (UID: \"d5878698-1651-4f18-b257-e5f0fa24e1d0\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:48:57.854255 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:57.854088    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:57.854255 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:57.854152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:48:57.854255 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:57.854187    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 04 22:48:57.854443 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:57.854264    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert podName:d5878698-1651-4f18-b257-e5f0fa24e1d0 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:01.854243656 +0000 UTC m=+229.964364240 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-ckz9h" (UID: "d5878698-1651-4f18-b257-e5f0fa24e1d0") : secret "networking-console-plugin-cert" not found
Sep 04 22:48:57.854443 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:57.854268    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 04 22:48:57.854443 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:57.854307    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle podName:fbbca652-c23c-4eaf-b908-7d724be0ab91 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:01.85428955 +0000 UTC m=+229.964410133 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle") pod "router-default-67c5f4dcb-cnpgt" (UID: "fbbca652-c23c-4eaf-b908-7d724be0ab91") : configmap references non-existent config key: service-ca.crt
Sep 04 22:48:57.854443 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:57.854327    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs podName:fbbca652-c23c-4eaf-b908-7d724be0ab91 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:01.854319803 +0000 UTC m=+229.964440367 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs") pod "router-default-67c5f4dcb-cnpgt" (UID: "fbbca652-c23c-4eaf-b908-7d724be0ab91") : secret "router-metrics-certs-default" not found
Sep 04 22:48:57.955475 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:57.955443    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:48:57.955632 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:57.955557    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:48:57.955632 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:57.955569    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77656967dd-bw4qw: secret "image-registry-tls" not found
Sep 04 22:48:57.955632 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:57.955616    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls podName:c98bf6bb-910a-4526-b463-7d6761159691 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:01.955604285 +0000 UTC m=+230.065724848 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls") pod "image-registry-77656967dd-bw4qw" (UID: "c98bf6bb-910a-4526-b463-7d6761159691") : secret "image-registry-tls" not found
Sep 04 22:48:57.994446 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:57.994398    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:48:58.104253 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.104152    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp" event={"ID":"0b1acb3a-ce94-4fa2-963b-f19969e1f344","Type":"ContainerStarted","Data":"5b433ba322715fb0d1204b584b17103966eeadfb446f194507f140b1816ddfea"}
Sep 04 22:48:58.105749 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.105723    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/0.log"
Sep 04 22:48:58.105872 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.105767    2578 generic.go:356] "Generic (PLEG): container finished" podID="6c89c14f-e6b5-4754-9182-64a5858d0e2e" containerID="a15a6d0834d74daa0a5bada481c6b1caa354cd1c080c69e2671c27e4abaae3cb" exitCode=255
Sep 04 22:48:58.105872 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.105815    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-58vxq" event={"ID":"6c89c14f-e6b5-4754-9182-64a5858d0e2e","Type":"ContainerDied","Data":"a15a6d0834d74daa0a5bada481c6b1caa354cd1c080c69e2671c27e4abaae3cb"}
Sep 04 22:48:58.106084 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.106033    2578 scope.go:117] "RemoveContainer" containerID="a15a6d0834d74daa0a5bada481c6b1caa354cd1c080c69e2671c27e4abaae3cb"
Sep 04 22:48:58.145750 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.145703    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp" podStartSLOduration=1.683787191 podStartE2EDuration="4.145686948s" totalImagesPullingTime="2.461899757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:48:58.12219268 +0000 UTC m=+226.232313267" watchObservedRunningTime="2026-09-04 22:48:58.145686948 +0000 UTC m=+226.255807534"
Sep 04 22:48:58.435555 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.435465    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:48:58.534495 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.534460    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:48:58.629446 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.629407    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:48:58.706432 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.706358    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:48:58.797225 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.797196    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:48:58.808170 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.808135    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8xsql"]
Sep 04 22:48:58.811262 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.811243    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:58.813742 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.813717    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 04 22:48:58.813975 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.813957    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 04 22:48:58.814686 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.814666    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 04 22:48:58.814792 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.814685    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pbtwn\""
Sep 04 22:48:58.814792 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.814688    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 04 22:48:58.824371 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.824350    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8xsql"]
Sep 04 22:48:58.864901 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.864875    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:48:58.965849 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.965773    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-crio-socket\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:58.965849 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.965831    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:58.966027 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.965940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:58.966027 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.965980    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfl28\" (UniqueName: \"kubernetes.io/projected/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-api-access-kfl28\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:58.966118 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:58.966045    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-data-volume\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:59.067385 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.067354    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:59.067495 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.067389    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfl28\" (UniqueName: \"kubernetes.io/projected/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-api-access-kfl28\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:59.067495 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.067442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-data-volume\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:59.067495 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.067473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-crio-socket\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:59.067495 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:59.067478    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 04 22:48:59.067705 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.067504    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:59.067705 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:59.067546    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-rbac-proxy-cm podName:8cd78b64-4ed2-4d53-87af-2fa41d0763ae nodeName:}" failed. No retries permitted until 2026-09-04 22:48:59.56752626 +0000 UTC m=+227.677646852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8xsql" (UID: "8cd78b64-4ed2-4d53-87af-2fa41d0763ae") : configmap "kube-rbac-proxy" not found
Sep 04 22:48:59.067705 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.067570    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-crio-socket\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:59.067705 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:59.067582    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 22:48:59.067705 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:59.067627    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls podName:8cd78b64-4ed2-4d53-87af-2fa41d0763ae nodeName:}" failed. No retries permitted until 2026-09-04 22:48:59.56761586 +0000 UTC m=+227.677736427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8xsql" (UID: "8cd78b64-4ed2-4d53-87af-2fa41d0763ae") : secret "insights-runtime-extractor-tls" not found
Sep 04 22:48:59.067968 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.067907    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-data-volume\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:59.075687 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.075664    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfl28\" (UniqueName: \"kubernetes.io/projected/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-api-access-kfl28\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:59.111063 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.111041    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 22:48:59.111428 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.111412    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/0.log"
Sep 04 22:48:59.111478 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.111453    2578 generic.go:356] "Generic (PLEG): container finished" podID="6c89c14f-e6b5-4754-9182-64a5858d0e2e" containerID="65a68c48c2aa7b4a52a337c7ba46ac1b0c2f33518a229f64a361d28f9be1a2d8" exitCode=255
Sep 04 22:48:59.111559 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.111539    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-58vxq" event={"ID":"6c89c14f-e6b5-4754-9182-64a5858d0e2e","Type":"ContainerDied","Data":"65a68c48c2aa7b4a52a337c7ba46ac1b0c2f33518a229f64a361d28f9be1a2d8"}
Sep 04 22:48:59.111620 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.111606    2578 scope.go:117] "RemoveContainer" containerID="a15a6d0834d74daa0a5bada481c6b1caa354cd1c080c69e2671c27e4abaae3cb"
Sep 04 22:48:59.111796 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.111780    2578 scope.go:117] "RemoveContainer" containerID="65a68c48c2aa7b4a52a337c7ba46ac1b0c2f33518a229f64a361d28f9be1a2d8"
Sep 04 22:48:59.111970 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:59.111950    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-58vxq_openshift-console-operator(6c89c14f-e6b5-4754-9182-64a5858d0e2e)\"" pod="openshift-console-operator/console-operator-85b454874b-58vxq" podUID="6c89c14f-e6b5-4754-9182-64a5858d0e2e"
Sep 04 22:48:59.572146 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.572107    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:59.572621 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:48:59.572205    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:48:59.572621 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:59.572248    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 04 22:48:59.572621 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:59.572286    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 22:48:59.572621 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:59.572333    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls podName:8cd78b64-4ed2-4d53-87af-2fa41d0763ae nodeName:}" failed. No retries permitted until 2026-09-04 22:49:00.572319945 +0000 UTC m=+228.682440509 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8xsql" (UID: "8cd78b64-4ed2-4d53-87af-2fa41d0763ae") : secret "insights-runtime-extractor-tls" not found
Sep 04 22:48:59.572621 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:48:59.572348    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-rbac-proxy-cm podName:8cd78b64-4ed2-4d53-87af-2fa41d0763ae nodeName:}" failed. No retries permitted until 2026-09-04 22:49:00.572342052 +0000 UTC m=+228.682462616 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8xsql" (UID: "8cd78b64-4ed2-4d53-87af-2fa41d0763ae") : configmap "kube-rbac-proxy" not found
Sep 04 22:49:00.116491 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:00.116459    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 22:49:00.116823 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:00.116805    2578 scope.go:117] "RemoveContainer" containerID="65a68c48c2aa7b4a52a337c7ba46ac1b0c2f33518a229f64a361d28f9be1a2d8"
Sep 04 22:49:00.116971 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:00.116952    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-58vxq_openshift-console-operator(6c89c14f-e6b5-4754-9182-64a5858d0e2e)\"" pod="openshift-console-operator/console-operator-85b454874b-58vxq" podUID="6c89c14f-e6b5-4754-9182-64a5858d0e2e"
Sep 04 22:49:00.341937 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:00.341905    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:00.409380 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:00.409305    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:00.579547 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:00.579508    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:49:00.579937 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:00.579567    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:49:00.579937 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:00.579675    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 22:49:00.579937 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:00.579692    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 04 22:49:00.579937 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:00.579743    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls podName:8cd78b64-4ed2-4d53-87af-2fa41d0763ae nodeName:}" failed. No retries permitted until 2026-09-04 22:49:02.579727416 +0000 UTC m=+230.689847981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8xsql" (UID: "8cd78b64-4ed2-4d53-87af-2fa41d0763ae") : secret "insights-runtime-extractor-tls" not found
Sep 04 22:49:00.579937 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:00.579760    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-rbac-proxy-cm podName:8cd78b64-4ed2-4d53-87af-2fa41d0763ae nodeName:}" failed. No retries permitted until 2026-09-04 22:49:02.579753675 +0000 UTC m=+230.689874239 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8xsql" (UID: "8cd78b64-4ed2-4d53-87af-2fa41d0763ae") : configmap "kube-rbac-proxy" not found
Sep 04 22:49:01.042460 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:01.042424    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:01.202498 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:01.202463    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:01.566653 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:01.566589    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k7nf5_7655d1a4-b050-4eea-b049-7c6899295630/dns-node-resolver/0.log"
Sep 04 22:49:01.890191 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:01.890155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ckz9h\" (UID: \"d5878698-1651-4f18-b257-e5f0fa24e1d0\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:49:01.890191 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:01.890200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:49:01.890717 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:01.890246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:49:01.890717 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:01.890318    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 04 22:49:01.890717 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:01.890345    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle podName:fbbca652-c23c-4eaf-b908-7d724be0ab91 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:09.890332192 +0000 UTC m=+238.000452756 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle") pod "router-default-67c5f4dcb-cnpgt" (UID: "fbbca652-c23c-4eaf-b908-7d724be0ab91") : configmap references non-existent config key: service-ca.crt
Sep 04 22:49:01.890717 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:01.890372    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert podName:d5878698-1651-4f18-b257-e5f0fa24e1d0 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:09.89035958 +0000 UTC m=+238.000480178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-ckz9h" (UID: "d5878698-1651-4f18-b257-e5f0fa24e1d0") : secret "networking-console-plugin-cert" not found
Sep 04 22:49:01.890717 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:01.890391    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 04 22:49:01.890717 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:01.890448    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs podName:fbbca652-c23c-4eaf-b908-7d724be0ab91 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:09.890432665 +0000 UTC m=+238.000553244 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs") pod "router-default-67c5f4dcb-cnpgt" (UID: "fbbca652-c23c-4eaf-b908-7d724be0ab91") : secret "router-metrics-certs-default" not found
Sep 04 22:49:01.991183 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:01.991148    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:49:01.991303 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:01.991284    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:49:01.991345 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:01.991305    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77656967dd-bw4qw: secret "image-registry-tls" not found
Sep 04 22:49:01.991377 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:01.991355    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls podName:c98bf6bb-910a-4526-b463-7d6761159691 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:09.99134007 +0000 UTC m=+238.101460634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls") pod "image-registry-77656967dd-bw4qw" (UID: "c98bf6bb-910a-4526-b463-7d6761159691") : secret "image-registry-tls" not found
Sep 04 22:49:02.366724 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:02.366695    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4qbrc_407039b1-b5f0-4538-8f92-f277e742b9d6/node-ca/0.log"
Sep 04 22:49:02.398732 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:02.398699    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:02.456208 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:02.456170    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:02.596434 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:02.596398    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:49:02.596579 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:02.596467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:49:02.596579 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:02.596550    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 22:49:02.596668 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:02.596609    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls podName:8cd78b64-4ed2-4d53-87af-2fa41d0763ae nodeName:}" failed. No retries permitted until 2026-09-04 22:49:06.5965941 +0000 UTC m=+234.706714681 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8xsql" (UID: "8cd78b64-4ed2-4d53-87af-2fa41d0763ae") : secret "insights-runtime-extractor-tls" not found
Sep 04 22:49:02.596935 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:02.596914    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:49:03.167816 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:03.167784    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hx25v_565701c0-684e-4fc0-9bc5-2ebce20c2100/serve-healthcheck-canary/0.log"
Sep 04 22:49:03.617808 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:03.617759    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:03.717715 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:03.717682    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:03.895886 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:03.895780    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:03.995317 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:03.995285    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:04.096754 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:04.096722    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:04.240465 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:04.240434    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:04.433849 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:04.433775    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:04.457782 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:04.457758    2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:49:04.457930 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:04.457807    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:49:04.458098 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:04.458087    2578 scope.go:117] "RemoveContainer" containerID="65a68c48c2aa7b4a52a337c7ba46ac1b0c2f33518a229f64a361d28f9be1a2d8"
Sep 04 22:49:04.458247 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:04.458231    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-58vxq_openshift-console-operator(6c89c14f-e6b5-4754-9182-64a5858d0e2e)\"" pod="openshift-console-operator/console-operator-85b454874b-58vxq" podUID="6c89c14f-e6b5-4754-9182-64a5858d0e2e"
Sep 04 22:49:05.137135 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:05.137106    2578 generic.go:356] "Generic (PLEG): container finished" podID="34fe39ee-e71f-4e93-98d5-c75a3c68fa2b" containerID="1f4218b8fdd3f84c01b2ee5d6f305466625b01f90bc984ca4d486c49b1c262b7" exitCode=255
Sep 04 22:49:05.137293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:05.137169    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w" event={"ID":"34fe39ee-e71f-4e93-98d5-c75a3c68fa2b","Type":"ContainerDied","Data":"1f4218b8fdd3f84c01b2ee5d6f305466625b01f90bc984ca4d486c49b1c262b7"}
Sep 04 22:49:05.142792 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:05.142770    2578 scope.go:117] "RemoveContainer" containerID="1f4218b8fdd3f84c01b2ee5d6f305466625b01f90bc984ca4d486c49b1c262b7"
Sep 04 22:49:05.619975 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:05.619941    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:05.824991 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:05.824962    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:06.020673 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:06.020576    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:06.142098 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:06.142066    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8fb4d48d-mxb6w" event={"ID":"34fe39ee-e71f-4e93-98d5-c75a3c68fa2b","Type":"ContainerStarted","Data":"c17489d85bfc8f0cb4f25e5732473a07ce73db697bbf1769a17308f165474fc5"}
Sep 04 22:49:06.217276 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:06.217238    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:06.415479 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:06.415444    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:06.627550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:06.627508    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:49:06.627948 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:06.627663    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 22:49:06.627948 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:06.627720    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls podName:8cd78b64-4ed2-4d53-87af-2fa41d0763ae nodeName:}" failed. No retries permitted until 2026-09-04 22:49:14.627706447 +0000 UTC m=+242.737827014 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8xsql" (UID: "8cd78b64-4ed2-4d53-87af-2fa41d0763ae") : secret "insights-runtime-extractor-tls" not found
Sep 04 22:49:07.022933 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:07.022902    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:07.216951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:07.216918    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:07.593138 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:07.593099    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:07.649945 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:07.649914    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:07.846181 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:07.846097    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:08.425237 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:08.425205    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:08.621033 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:08.620999    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:08.818865 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:08.818784    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:09.055879 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:09.055845    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:09.219200 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:09.219167    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:09.954396 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:09.954354    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ckz9h\" (UID: \"d5878698-1651-4f18-b257-e5f0fa24e1d0\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:49:09.954396 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:09.954403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:49:09.954957 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:09.954525    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:49:09.954957 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:09.954613    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 04 22:49:09.954957 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:09.954700    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle podName:fbbca652-c23c-4eaf-b908-7d724be0ab91 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:25.954680765 +0000 UTC m=+254.064801345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle") pod "router-default-67c5f4dcb-cnpgt" (UID: "fbbca652-c23c-4eaf-b908-7d724be0ab91") : configmap references non-existent config key: service-ca.crt
Sep 04 22:49:09.954957 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:09.954718    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert podName:d5878698-1651-4f18-b257-e5f0fa24e1d0 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:25.954709486 +0000 UTC m=+254.064830057 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-ckz9h" (UID: "d5878698-1651-4f18-b257-e5f0fa24e1d0") : secret "networking-console-plugin-cert" not found
Sep 04 22:49:09.956850 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:09.956826    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbbca652-c23c-4eaf-b908-7d724be0ab91-metrics-certs\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:49:10.022596 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:10.022562    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:10.054966 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:10.054938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:49:10.057257 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:10.057239    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls\") pod \"image-registry-77656967dd-bw4qw\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") " pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:49:10.069783 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:10.069758    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:49:10.122966 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:10.122893    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:10.341348 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:10.341314    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77656967dd-bw4qw"]
Sep 04 22:49:10.344210 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:49:10.344186    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98bf6bb_910a_4526_b463_7d6761159691.slice/crio-01c7ec39f4ce9e12e517b2814b145dc76e857c7f84926a50e051f0f224c6ae9a WatchSource:0}: Error finding container 01c7ec39f4ce9e12e517b2814b145dc76e857c7f84926a50e051f0f224c6ae9a: Status 404 returned error can't find the container with id 01c7ec39f4ce9e12e517b2814b145dc76e857c7f84926a50e051f0f224c6ae9a
Sep 04 22:49:10.431993 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:10.431962    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:10.818506 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:10.818420    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:11.020353 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:11.020321    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:11.164152 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:11.164115    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77656967dd-bw4qw" event={"ID":"c98bf6bb-910a-4526-b463-7d6761159691","Type":"ContainerStarted","Data":"2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38"}
Sep 04 22:49:11.164356 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:11.164178    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77656967dd-bw4qw" event={"ID":"c98bf6bb-910a-4526-b463-7d6761159691","Type":"ContainerStarted","Data":"01c7ec39f4ce9e12e517b2814b145dc76e857c7f84926a50e051f0f224c6ae9a"}
Sep 04 22:49:11.164356 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:11.164234    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:49:11.183411 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:11.183352    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77656967dd-bw4qw" podStartSLOduration=17.183335806 podStartE2EDuration="17.183335806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:48:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:11.182123757 +0000 UTC m=+239.292244344" watchObservedRunningTime="2026-09-04 22:49:11.183335806 +0000 UTC m=+239.293456391"
Sep 04 22:49:11.633355 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:11.633326    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:12.312505 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:12.312463    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:12.417677 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:12.417633    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:12.822099 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:12.822058    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:13.017837 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:13.017782    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:14.617457 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:14.617425    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:14.690373 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:14.690336    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:49:14.692471 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:14.692451    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cd78b64-4ed2-4d53-87af-2fa41d0763ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xsql\" (UID: \"8cd78b64-4ed2-4d53-87af-2fa41d0763ae\") " pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:49:14.721432 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:14.721397    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pbtwn\""
Sep 04 22:49:14.730340 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:14.730313    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8xsql"
Sep 04 22:49:15.004263 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:15.004190    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8xsql"]
Sep 04 22:49:15.007793 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:49:15.007768    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cd78b64_4ed2_4d53_87af_2fa41d0763ae.slice/crio-979020cc11bcd758b494c982b1f2a8ec9d9fa670c64d17672e1d5d1cb60e76cb WatchSource:0}: Error finding container 979020cc11bcd758b494c982b1f2a8ec9d9fa670c64d17672e1d5d1cb60e76cb: Status 404 returned error can't find the container with id 979020cc11bcd758b494c982b1f2a8ec9d9fa670c64d17672e1d5d1cb60e76cb
Sep 04 22:49:15.177338 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:15.177305    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8xsql" event={"ID":"8cd78b64-4ed2-4d53-87af-2fa41d0763ae","Type":"ContainerStarted","Data":"68a2cc5dd7ecb901d2c69985a229767e056198fb31199727752d200b544950dd"}
Sep 04 22:49:15.177477 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:15.177362    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8xsql" event={"ID":"8cd78b64-4ed2-4d53-87af-2fa41d0763ae","Type":"ContainerStarted","Data":"979020cc11bcd758b494c982b1f2a8ec9d9fa670c64d17672e1d5d1cb60e76cb"}
Sep 04 22:49:16.182017 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:16.181981    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8xsql" event={"ID":"8cd78b64-4ed2-4d53-87af-2fa41d0763ae","Type":"ContainerStarted","Data":"40dce4744a748dbc728b71d79ec7f411cbe5c80870d8873fd766c7f22d7cca22"}
Sep 04 22:49:17.376123 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:17.376086    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:17.413442 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:17.413373    2578 scope.go:117] "RemoveContainer" containerID="65a68c48c2aa7b4a52a337c7ba46ac1b0c2f33518a229f64a361d28f9be1a2d8"
Sep 04 22:49:17.476096 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:17.476074    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:18.189512 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:18.189473    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8xsql" event={"ID":"8cd78b64-4ed2-4d53-87af-2fa41d0763ae","Type":"ContainerStarted","Data":"60a8a35fc1d5eb5b270775d2157b2ffe0b4809c9e0aef30e04a508a352d21967"}
Sep 04 22:49:18.191273 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:18.191254    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 22:49:18.191382 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:18.191310    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-58vxq" event={"ID":"6c89c14f-e6b5-4754-9182-64a5858d0e2e","Type":"ContainerStarted","Data":"02270f046c9b4f928ec0b4f5ec97a5b00e0fb5e5557f46c2415a4ca6e2ea7b98"}
Sep 04 22:49:18.191559 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:18.191540    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:49:18.205895 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:18.205853    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8xsql" podStartSLOduration=18.190257576 podStartE2EDuration="20.205838539s" totalImagesPullingTime="2.015580963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:48:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:18.204631269 +0000 UTC m=+246.314751856" watchObservedRunningTime="2026-09-04 22:49:18.205838539 +0000 UTC m=+246.315959125"
Sep 04 22:49:18.220150 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:18.220109    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-58vxq" podStartSLOduration=21.782501131 podStartE2EDuration="24.220095209s" totalImagesPullingTime="2.437594078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:18.219516314 +0000 UTC m=+246.329636903" watchObservedRunningTime="2026-09-04 22:49:18.220095209 +0000 UTC m=+246.330215796"
Sep 04 22:49:18.432239 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:18.432200    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:18.489369 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:18.489261    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:18.611666 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:18.611622    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-58vxq"
Sep 04 22:49:18.801554 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:18.801472    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:18.902996 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:18.902963    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:19.003695 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:19.003661    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:19.294657 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:19.294606    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:19.468175 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:19.468138    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:19.522546 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:19.522516    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:19.662466 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:19.662433    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:19.718096 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:19.718058    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:19.865753 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:19.865719    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:19.920213 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:19.920125    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:20.280615 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:20.280542    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:20.885505 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:20.885477    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:20.986592 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:20.986562    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:21.497896 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:21.497864    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:21.554349 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:21.554322    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:22.988652 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:22.988606    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77656967dd-bw4qw"]
Sep 04 22:49:23.049617 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:23.049584    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:23.136411 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:23.136380    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:23.239360 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:23.239274    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:24.265942 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:24.265896    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:49:24.268302 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:24.268278    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ec67b5-ff79-49d1-b063-8cd7c4a56c84-metrics-certs\") pod \"network-metrics-daemon-9dqmt\" (UID: \"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84\") " pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:49:24.416968 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:24.416937    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sxf76\""
Sep 04 22:49:24.424879 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:24.424860    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dqmt"
Sep 04 22:49:24.690258 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:24.690236    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9dqmt"]
Sep 04 22:49:24.692909 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:49:24.692882    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ec67b5_ff79_49d1_b063_8cd7c4a56c84.slice/crio-d2203c47d0bdcf032d1c4dd9854936294358eefdadb1a51e12ec20844f41ff9f WatchSource:0}: Error finding container d2203c47d0bdcf032d1c4dd9854936294358eefdadb1a51e12ec20844f41ff9f: Status 404 returned error can't find the container with id d2203c47d0bdcf032d1c4dd9854936294358eefdadb1a51e12ec20844f41ff9f
Sep 04 22:49:25.212520 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:25.212478    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9dqmt" event={"ID":"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84","Type":"ContainerStarted","Data":"d2203c47d0bdcf032d1c4dd9854936294358eefdadb1a51e12ec20844f41ff9f"}
Sep 04 22:49:25.980453 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:25.980415    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ckz9h\" (UID: \"d5878698-1651-4f18-b257-e5f0fa24e1d0\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:49:25.980842 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:25.980467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:49:25.981044 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:25.981026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbbca652-c23c-4eaf-b908-7d724be0ab91-service-ca-bundle\") pod \"router-default-67c5f4dcb-cnpgt\" (UID: \"fbbca652-c23c-4eaf-b908-7d724be0ab91\") " pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:49:25.982752 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:25.982735    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5878698-1651-4f18-b257-e5f0fa24e1d0-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ckz9h\" (UID: \"d5878698-1651-4f18-b257-e5f0fa24e1d0\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:49:26.216961 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:26.216929    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9dqmt" event={"ID":"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84","Type":"ContainerStarted","Data":"235354fe62145848e6b601151856922440c0480572ac8b0c6ad0a76c970b2523"}
Sep 04 22:49:26.217111 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:26.216983    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9dqmt" event={"ID":"a7ec67b5-ff79-49d1-b063-8cd7c4a56c84","Type":"ContainerStarted","Data":"d01ff41fd98d2086f63b53d0d620175c36bc170cc60fc9c5ff87b7738f89c698"}
Sep 04 22:49:26.232320 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:26.232228    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9dqmt" podStartSLOduration=252.400257968 podStartE2EDuration="4m13.232213064s" totalImagesPullingTime="831.955096ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:26.230869352 +0000 UTC m=+254.340989938" watchObservedRunningTime="2026-09-04 22:49:26.232213064 +0000 UTC m=+254.342333649"
Sep 04 22:49:26.243737 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:26.243714    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-m2w6p\""
Sep 04 22:49:26.251558 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:26.251540    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h"
Sep 04 22:49:26.265799 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:26.265778    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f7cw8\""
Sep 04 22:49:26.273678 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:26.273656    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:49:26.311622 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:26.311554    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:26.368812 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:26.368782    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:26.518927 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:26.518848    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-ckz9h"]
Sep 04 22:49:26.522079 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:49:26.522056    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5878698_1651_4f18_b257_e5f0fa24e1d0.slice/crio-9e1b861f65a67d80123905b219adeedc53c6e0b706e0aa0ad21ac5773f4cad5c WatchSource:0}: Error finding container 9e1b861f65a67d80123905b219adeedc53c6e0b706e0aa0ad21ac5773f4cad5c: Status 404 returned error can't find the container with id 9e1b861f65a67d80123905b219adeedc53c6e0b706e0aa0ad21ac5773f4cad5c
Sep 04 22:49:26.549275 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:26.549248    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-67c5f4dcb-cnpgt"]
Sep 04 22:49:26.552921 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:49:26.552897    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbbca652_c23c_4eaf_b908_7d724be0ab91.slice/crio-3ca4fbe4afc003e53c5e2f74cba1542be03899a42c773cfc84856225c8aafe43 WatchSource:0}: Error finding container 3ca4fbe4afc003e53c5e2f74cba1542be03899a42c773cfc84856225c8aafe43: Status 404 returned error can't find the container with id 3ca4fbe4afc003e53c5e2f74cba1542be03899a42c773cfc84856225c8aafe43
Sep 04 22:49:27.222379 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:27.222338    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-67c5f4dcb-cnpgt" event={"ID":"fbbca652-c23c-4eaf-b908-7d724be0ab91","Type":"ContainerStarted","Data":"cd2be2e4fb3f9db16333058224b990513bb2ae1056e3f7103726acaa7f953b09"}
Sep 04 22:49:27.222866 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:27.222407    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-67c5f4dcb-cnpgt" event={"ID":"fbbca652-c23c-4eaf-b908-7d724be0ab91","Type":"ContainerStarted","Data":"3ca4fbe4afc003e53c5e2f74cba1542be03899a42c773cfc84856225c8aafe43"}
Sep 04 22:49:27.223804 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:27.223771    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h" event={"ID":"d5878698-1651-4f18-b257-e5f0fa24e1d0","Type":"ContainerStarted","Data":"9e1b861f65a67d80123905b219adeedc53c6e0b706e0aa0ad21ac5773f4cad5c"}
Sep 04 22:49:27.242897 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:27.242851    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-67c5f4dcb-cnpgt" podStartSLOduration=33.242833266 podStartE2EDuration="33.242833266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:48:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:27.240429778 +0000 UTC m=+255.350550363" watchObservedRunningTime="2026-09-04 22:49:27.242833266 +0000 UTC m=+255.352953853"
Sep 04 22:49:27.274086 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:27.274049    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:49:27.276933 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:27.276909    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:49:28.228899 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:28.228852    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h" event={"ID":"d5878698-1651-4f18-b257-e5f0fa24e1d0","Type":"ContainerStarted","Data":"d9003e519495257cadeac81c2ebffa84cbab6718719f40fe7cf69f7eff27c4bb"}
Sep 04 22:49:28.229312 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:28.229090    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:49:28.230204 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:28.230179    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-67c5f4dcb-cnpgt"
Sep 04 22:49:28.243931 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:28.243889    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-ckz9h" podStartSLOduration=33.044401271 podStartE2EDuration="34.243877473s" totalImagesPullingTime="1.199476202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:28.24367612 +0000 UTC m=+256.353796701" watchObservedRunningTime="2026-09-04 22:49:28.243877473 +0000 UTC m=+256.353998060"
Sep 04 22:49:28.339841 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:28.339812    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:28.441901 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:28.441869    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:28.629569 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:28.629540    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:28.732352 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:28.732317    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:28.806384 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:28.806356    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:29.359974 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:29.359942    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:29.461216 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:29.461178    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:29.515392 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:29.515367    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:32.390591 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.390559    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:32.458036 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.457999    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:32.818339 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.818250    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:32.853226 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.853198    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"]
Sep 04 22:49:32.857347 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.857329    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:32.860666 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.860632    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 04 22:49:32.860781 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.860667    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 04 22:49:32.860781 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.860674    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 04 22:49:32.860781 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.860715    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 04 22:49:32.860781 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.860715    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 04 22:49:32.860781 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.860769    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-q7gqp\""
Sep 04 22:49:32.868756 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.867076    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"]
Sep 04 22:49:32.883261 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.883238    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:32.934667 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.934607    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb50d22d-682c-47cd-a778-c7c6af5cc95e-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:32.934803 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.934675    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2mbx\" (UniqueName: \"kubernetes.io/projected/eb50d22d-682c-47cd-a778-c7c6af5cc95e-kube-api-access-m2mbx\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:32.934803 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.934725    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb50d22d-682c-47cd-a778-c7c6af5cc95e-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:32.934803 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.934757    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb50d22d-682c-47cd-a778-c7c6af5cc95e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:32.994451 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.994418    2578 patch_prober.go:28] interesting pod/image-registry-77656967dd-bw4qw container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 04 22:49:32.994602 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:32.994478    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-77656967dd-bw4qw" podUID="c98bf6bb-910a-4526-b463-7d6761159691" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 04 22:49:33.035807 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:33.035776    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2mbx\" (UniqueName: \"kubernetes.io/projected/eb50d22d-682c-47cd-a778-c7c6af5cc95e-kube-api-access-m2mbx\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:33.035963 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:33.035832    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb50d22d-682c-47cd-a778-c7c6af5cc95e-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:33.035963 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:33.035862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb50d22d-682c-47cd-a778-c7c6af5cc95e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:33.035963 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:33.035888    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb50d22d-682c-47cd-a778-c7c6af5cc95e-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:33.036052 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:33.035990    2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 04 22:49:33.036085 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:33.036053    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb50d22d-682c-47cd-a778-c7c6af5cc95e-prometheus-operator-tls podName:eb50d22d-682c-47cd-a778-c7c6af5cc95e nodeName:}" failed. No retries permitted until 2026-09-04 22:49:33.536036145 +0000 UTC m=+261.646156710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/eb50d22d-682c-47cd-a778-c7c6af5cc95e-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-dpv8r" (UID: "eb50d22d-682c-47cd-a778-c7c6af5cc95e") : secret "prometheus-operator-tls" not found
Sep 04 22:49:33.036585 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:33.036563    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb50d22d-682c-47cd-a778-c7c6af5cc95e-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:33.038258 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:33.038232    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb50d22d-682c-47cd-a778-c7c6af5cc95e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:33.044131 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:33.044104    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2mbx\" (UniqueName: \"kubernetes.io/projected/eb50d22d-682c-47cd-a778-c7c6af5cc95e-kube-api-access-m2mbx\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:33.325823 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:33.325787    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:33.540102 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:33.540063    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb50d22d-682c-47cd-a778-c7c6af5cc95e-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:33.542519 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:33.542497    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb50d22d-682c-47cd-a778-c7c6af5cc95e-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-dpv8r\" (UID: \"eb50d22d-682c-47cd-a778-c7c6af5cc95e\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:33.768022 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:33.767930    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"
Sep 04 22:49:34.034701 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.034581    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-dpv8r"]
Sep 04 22:49:34.038908 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:49:34.038875    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb50d22d_682c_47cd_a778_c7c6af5cc95e.slice/crio-d25ea95dbc79806e059eec840b9410dc91f71ceeec49440e044abd92e2900739 WatchSource:0}: Error finding container d25ea95dbc79806e059eec840b9410dc91f71ceeec49440e044abd92e2900739: Status 404 returned error can't find the container with id d25ea95dbc79806e059eec840b9410dc91f71ceeec49440e044abd92e2900739
Sep 04 22:49:34.249686 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.249632    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r" event={"ID":"eb50d22d-682c-47cd-a778-c7c6af5cc95e","Type":"ContainerStarted","Data":"d25ea95dbc79806e059eec840b9410dc91f71ceeec49440e044abd92e2900739"}
Sep 04 22:49:34.509678 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.509633    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58f67c784b-zrpfj"]
Sep 04 22:49:34.513835 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.513818    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.516360 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.516337    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 04 22:49:34.517214 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.517197    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nrl48\""
Sep 04 22:49:34.517285 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.517236    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 04 22:49:34.517285 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.517234    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 04 22:49:34.517385 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.517304    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 04 22:49:34.517385 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.517318    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 04 22:49:34.517385 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.517334    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 04 22:49:34.517587 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.517572    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 04 22:49:34.522158 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.521858    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58f67c784b-zrpfj"]
Sep 04 22:49:34.650703 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.650672    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxjbl\" (UniqueName: \"kubernetes.io/projected/8876e514-4ff7-4b06-97f5-b64fea52173f-kube-api-access-gxjbl\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.650703 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.650709    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-oauth-serving-cert\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.651095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.650727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8876e514-4ff7-4b06-97f5-b64fea52173f-console-serving-cert\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.651095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.650762    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-console-config\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.651095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.650784    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-service-ca\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.651095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.650814    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8876e514-4ff7-4b06-97f5-b64fea52173f-console-oauth-config\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.752069 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.752029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-console-config\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.752069 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.752070    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-service-ca\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.752345 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.752112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8876e514-4ff7-4b06-97f5-b64fea52173f-console-oauth-config\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.752345 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.752176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxjbl\" (UniqueName: \"kubernetes.io/projected/8876e514-4ff7-4b06-97f5-b64fea52173f-kube-api-access-gxjbl\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.752345 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.752211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-oauth-serving-cert\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.752345 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.752234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8876e514-4ff7-4b06-97f5-b64fea52173f-console-serving-cert\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.752898 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.752863    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-console-config\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.753009 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.752878    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-service-ca\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.753331 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.753308    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-oauth-serving-cert\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.754816 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.754774    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8876e514-4ff7-4b06-97f5-b64fea52173f-console-oauth-config\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.754913 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.754897    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8876e514-4ff7-4b06-97f5-b64fea52173f-console-serving-cert\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.760053 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.760005    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxjbl\" (UniqueName: \"kubernetes.io/projected/8876e514-4ff7-4b06-97f5-b64fea52173f-kube-api-access-gxjbl\") pod \"console-58f67c784b-zrpfj\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") " pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:34.824039 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:34.824001    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:35.101575 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:35.101539    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58f67c784b-zrpfj"]
Sep 04 22:49:35.350788 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:49:35.350754    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8876e514_4ff7_4b06_97f5_b64fea52173f.slice/crio-c67bc6ca40ced40685d0b4d45b30397286c202866d0907049ded316b295b421d WatchSource:0}: Error finding container c67bc6ca40ced40685d0b4d45b30397286c202866d0907049ded316b295b421d: Status 404 returned error can't find the container with id c67bc6ca40ced40685d0b4d45b30397286c202866d0907049ded316b295b421d
Sep 04 22:49:35.513440 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:35.513415    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:35.615590 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:35.615505    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:35.714684 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:35.714628    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:36.260113 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:36.260074    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f67c784b-zrpfj" event={"ID":"8876e514-4ff7-4b06-97f5-b64fea52173f","Type":"ContainerStarted","Data":"c67bc6ca40ced40685d0b4d45b30397286c202866d0907049ded316b295b421d"}
Sep 04 22:49:36.262309 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:36.262280    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r" event={"ID":"eb50d22d-682c-47cd-a778-c7c6af5cc95e","Type":"ContainerStarted","Data":"a533a921ffcd672bf4d8b5f101e1482c5e4d1f3f87a8c01ed5e5eac283b26536"}
Sep 04 22:49:36.262464 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:36.262328    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r" event={"ID":"eb50d22d-682c-47cd-a778-c7c6af5cc95e","Type":"ContainerStarted","Data":"adbb6cf119a3a20f11c84481d3fef45d893a3a19929a6281e64d5897a052e362"}
Sep 04 22:49:36.278463 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:36.278410    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-dpv8r" podStartSLOduration=2.924694133 podStartE2EDuration="4.278392414s" totalImagesPullingTime="1.353698281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:36.277759739 +0000 UTC m=+264.387880326" watchObservedRunningTime="2026-09-04 22:49:36.278392414 +0000 UTC m=+264.388513001"
Sep 04 22:49:36.701495 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:36.701461    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:37.947393 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:37.947362    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:38.158406 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.158359    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:38.207474 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.207395    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"]
Sep 04 22:49:38.216429 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.216403    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.217394 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.217367    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-p6g4x"]
Sep 04 22:49:38.218858 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.218832    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 04 22:49:38.219613 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.219592    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-w5w8x\""
Sep 04 22:49:38.219753 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.219592    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 04 22:49:38.219753 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.219614    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 04 22:49:38.220653 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.220624    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.223481 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.223455    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 04 22:49:38.223481 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.223473    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 04 22:49:38.223826 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.223746    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tvwbx\""
Sep 04 22:49:38.223935 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.223841    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"]
Sep 04 22:49:38.224010 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.223990    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 04 22:49:38.244619 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.244596    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:38.312935 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.312911    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:38.384101 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384059    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.384101 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384097    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c3a1a96-67cc-4068-8bcd-b949c651fd26-metrics-client-ca\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.384330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz86t\" (UniqueName: \"kubernetes.io/projected/1c3a1a96-67cc-4068-8bcd-b949c651fd26-kube-api-access-xz86t\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.384330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384132    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e05e4642-b09c-4900-a384-f433d0f6188e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.384330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384185    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e05e4642-b09c-4900-a384-f433d0f6188e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.384330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384214    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-wtmp\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.384330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384254    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-tls\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.384330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384273    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c3a1a96-67cc-4068-8bcd-b949c651fd26-sys\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.384330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384303    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c3a1a96-67cc-4068-8bcd-b949c651fd26-root\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.384330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-textfile\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.384776 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384342    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-accelerators-collector-config\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.384776 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384358    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr55p\" (UniqueName: \"kubernetes.io/projected/e05e4642-b09c-4900-a384-f433d0f6188e-kube-api-access-lr55p\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.384776 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384387    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e05e4642-b09c-4900-a384-f433d0f6188e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.384776 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384408    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e05e4642-b09c-4900-a384-f433d0f6188e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.384776 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.384428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e05e4642-b09c-4900-a384-f433d0f6188e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.470658 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.470547    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:38.485609 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485583    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-tls\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.485719 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485614    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c3a1a96-67cc-4068-8bcd-b949c651fd26-sys\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.485719 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c3a1a96-67cc-4068-8bcd-b949c651fd26-root\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.485719 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485686    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-textfile\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.485719 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-accelerators-collector-config\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.485907 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr55p\" (UniqueName: \"kubernetes.io/projected/e05e4642-b09c-4900-a384-f433d0f6188e-kube-api-access-lr55p\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.485907 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485741    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c3a1a96-67cc-4068-8bcd-b949c651fd26-sys\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.485907 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485713    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c3a1a96-67cc-4068-8bcd-b949c651fd26-root\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.485907 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485773    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e05e4642-b09c-4900-a384-f433d0f6188e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.485907 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485833    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e05e4642-b09c-4900-a384-f433d0f6188e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.485907 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e05e4642-b09c-4900-a384-f433d0f6188e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.486182 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485927    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.486182 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485963    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c3a1a96-67cc-4068-8bcd-b949c651fd26-metrics-client-ca\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.486182 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.485992    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz86t\" (UniqueName: \"kubernetes.io/projected/1c3a1a96-67cc-4068-8bcd-b949c651fd26-kube-api-access-xz86t\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.486182 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.486015    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e05e4642-b09c-4900-a384-f433d0f6188e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.486182 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.486048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e05e4642-b09c-4900-a384-f433d0f6188e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.486182 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.486054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-textfile\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.486182 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.486077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-wtmp\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.486512 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.486244    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-wtmp\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.486512 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.486417    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-accelerators-collector-config\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.486615 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:38.486516    2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 04 22:49:38.486615 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:38.486572    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e05e4642-b09c-4900-a384-f433d0f6188e-kube-state-metrics-tls podName:e05e4642-b09c-4900-a384-f433d0f6188e nodeName:}" failed. No retries permitted until 2026-09-04 22:49:38.98655329 +0000 UTC m=+267.096673863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e05e4642-b09c-4900-a384-f433d0f6188e-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-2vhtw" (UID: "e05e4642-b09c-4900-a384-f433d0f6188e") : secret "kube-state-metrics-tls" not found
Sep 04 22:49:38.486759 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.486627    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e05e4642-b09c-4900-a384-f433d0f6188e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.486903 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.486883    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e05e4642-b09c-4900-a384-f433d0f6188e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.486990 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.486966    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e05e4642-b09c-4900-a384-f433d0f6188e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.487025 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.487010    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c3a1a96-67cc-4068-8bcd-b949c651fd26-metrics-client-ca\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.488203 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.488180    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-tls\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.488716 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.488694    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c3a1a96-67cc-4068-8bcd-b949c651fd26-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.488716 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.488713    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e05e4642-b09c-4900-a384-f433d0f6188e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.496228 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.496199    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz86t\" (UniqueName: \"kubernetes.io/projected/1c3a1a96-67cc-4068-8bcd-b949c651fd26-kube-api-access-xz86t\") pod \"node-exporter-p6g4x\" (UID: \"1c3a1a96-67cc-4068-8bcd-b949c651fd26\") " pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.497271 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.497245    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr55p\" (UniqueName: \"kubernetes.io/projected/e05e4642-b09c-4900-a384-f433d0f6188e-kube-api-access-lr55p\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.535592 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.535555    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p6g4x"
Sep 04 22:49:38.546951 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:49:38.546923    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c3a1a96_67cc_4068_8bcd_b949c651fd26.slice/crio-3f6d7e9c0fc16ab1fc610112cbcdf140da82115a93eb828e036d8563429bf207 WatchSource:0}: Error finding container 3f6d7e9c0fc16ab1fc610112cbcdf140da82115a93eb828e036d8563429bf207: Status 404 returned error can't find the container with id 3f6d7e9c0fc16ab1fc610112cbcdf140da82115a93eb828e036d8563429bf207
Sep 04 22:49:38.571176 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.571141    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:38.671145 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.671111    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:38.941959 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.941924    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:38.991824 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.991789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e05e4642-b09c-4900-a384-f433d0f6188e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:38.994149 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:38.994129    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e05e4642-b09c-4900-a384-f433d0f6188e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-2vhtw\" (UID: \"e05e4642-b09c-4900-a384-f433d0f6188e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:39.041539 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.041507    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:39.129539 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.129508    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"
Sep 04 22:49:39.166435 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.166401    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:39.242707 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.242618    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:39.247388 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.247358    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 22:49:39.251922 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.251898    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.254423 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.254397    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 04 22:49:39.254544 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.254444    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 04 22:49:39.254544 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.254479    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 04 22:49:39.254544 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.254501    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 04 22:49:39.255028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.254548    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jbw85\""
Sep 04 22:49:39.255028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.254570    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 04 22:49:39.255028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.254811    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 04 22:49:39.255028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.254866    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 04 22:49:39.255028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.254873    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 04 22:49:39.255028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.254870    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 04 22:49:39.268875 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.268847    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 22:49:39.275420 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.275390    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p6g4x" event={"ID":"1c3a1a96-67cc-4068-8bcd-b949c651fd26","Type":"ContainerStarted","Data":"3f6d7e9c0fc16ab1fc610112cbcdf140da82115a93eb828e036d8563429bf207"}
Sep 04 22:49:39.277028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.276997    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f67c784b-zrpfj" event={"ID":"8876e514-4ff7-4b06-97f5-b64fea52173f","Type":"ContainerStarted","Data":"065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945"}
Sep 04 22:49:39.308509 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.308445    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58f67c784b-zrpfj" podStartSLOduration=2.359225004 podStartE2EDuration="5.308423459s" totalImagesPullingTime="2.949198455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:39.307083569 +0000 UTC m=+267.417204156" watchObservedRunningTime="2026-09-04 22:49:39.308423459 +0000 UTC m=+267.418544046"
Sep 04 22:49:39.310982 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.310957    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:39.379658 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.379615    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:39.394188 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.394150    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.394342 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.394196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.394342 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.394218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/18c8b332-4fff-4ca8-96f3-3fbd55096106-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.394342 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.394272    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-config-volume\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.394342 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.394337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/18c8b332-4fff-4ca8-96f3-3fbd55096106-tls-assets\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.394561 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.394488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x98x\" (UniqueName: \"kubernetes.io/projected/18c8b332-4fff-4ca8-96f3-3fbd55096106-kube-api-access-2x98x\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.394611 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.394558    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.394699 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.394667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/18c8b332-4fff-4ca8-96f3-3fbd55096106-config-out\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.394699 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.394694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.394799 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.394719    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.395120 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.395095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c8b332-4fff-4ca8-96f3-3fbd55096106-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.395248 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.395231    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18c8b332-4fff-4ca8-96f3-3fbd55096106-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.395313 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.395284    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-web-config\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.409835 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.409463    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw"]
Sep 04 22:49:39.478654 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:49:39.478607    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode05e4642_b09c_4900_a384_f433d0f6188e.slice/crio-a78c2afca1602f26c05097934e673987add3b636f001b8b7e63eb0540b4bda64 WatchSource:0}: Error finding container a78c2afca1602f26c05097934e673987add3b636f001b8b7e63eb0540b4bda64: Status 404 returned error can't find the container with id a78c2afca1602f26c05097934e673987add3b636f001b8b7e63eb0540b4bda64
Sep 04 22:49:39.495978 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.495923    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.495978 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.495954    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.495978 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.495971    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/18c8b332-4fff-4ca8-96f3-3fbd55096106-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.496178 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.495987    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-config-volume\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.496178 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.496014    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/18c8b332-4fff-4ca8-96f3-3fbd55096106-tls-assets\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.496178 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.496044    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x98x\" (UniqueName: \"kubernetes.io/projected/18c8b332-4fff-4ca8-96f3-3fbd55096106-kube-api-access-2x98x\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.496178 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.496106    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.496178 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.496152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/18c8b332-4fff-4ca8-96f3-3fbd55096106-config-out\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.496427 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.496179    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.496427 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.496202    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.496427 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.496249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c8b332-4fff-4ca8-96f3-3fbd55096106-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.496427 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.496282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18c8b332-4fff-4ca8-96f3-3fbd55096106-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.496427 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.496315    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-web-config\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.496427 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.496410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/18c8b332-4fff-4ca8-96f3-3fbd55096106-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.498145 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.498113    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c8b332-4fff-4ca8-96f3-3fbd55096106-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.498884 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.498839    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18c8b332-4fff-4ca8-96f3-3fbd55096106-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.499064 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.499039    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.499420 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.499375    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-config-volume\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.499501 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.499440    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.499501 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.499473    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-web-config\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.499586 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.499563    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/18c8b332-4fff-4ca8-96f3-3fbd55096106-config-out\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.499801 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.499784    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.500407 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.500387    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/18c8b332-4fff-4ca8-96f3-3fbd55096106-tls-assets\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.501161 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.501139    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.501308 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.501292    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.506144 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.506120    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x98x\" (UniqueName: \"kubernetes.io/projected/18c8b332-4fff-4ca8-96f3-3fbd55096106-kube-api-access-2x98x\") pod \"alertmanager-main-0\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.562385 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.562354    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:49:39.570030 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.570001    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:39.669808 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.669780    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:39.844705 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:39.844669    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 22:49:39.848193 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:49:39.848158    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c8b332_4fff_4ca8_96f3_3fbd55096106.slice/crio-e943d73cf690a160b419a4491b1f487d018921a674413c36c89fa2f9893389c8 WatchSource:0}: Error finding container e943d73cf690a160b419a4491b1f487d018921a674413c36c89fa2f9893389c8: Status 404 returned error can't find the container with id e943d73cf690a160b419a4491b1f487d018921a674413c36c89fa2f9893389c8
Sep 04 22:49:40.282782 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:40.282693    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerStarted","Data":"e943d73cf690a160b419a4491b1f487d018921a674413c36c89fa2f9893389c8"}
Sep 04 22:49:40.284506 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:40.284471    2578 generic.go:356] "Generic (PLEG): container finished" podID="1c3a1a96-67cc-4068-8bcd-b949c651fd26" containerID="570ddecd10b25e76120a6f833ecdfeac8aac643a94b44df124281a1eeb9da4d8" exitCode=0
Sep 04 22:49:40.284626 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:40.284516    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p6g4x" event={"ID":"1c3a1a96-67cc-4068-8bcd-b949c651fd26","Type":"ContainerDied","Data":"570ddecd10b25e76120a6f833ecdfeac8aac643a94b44df124281a1eeb9da4d8"}
Sep 04 22:49:40.286307 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:40.286281    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw" event={"ID":"e05e4642-b09c-4900-a384-f433d0f6188e","Type":"ContainerStarted","Data":"a78c2afca1602f26c05097934e673987add3b636f001b8b7e63eb0540b4bda64"}
Sep 04 22:49:41.146462 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:41.146436    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:41.230750 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:41.230717    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:41.292547 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:41.292518    2578 generic.go:356] "Generic (PLEG): container finished" podID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerID="18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca" exitCode=0
Sep 04 22:49:41.292947 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:41.292659    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerDied","Data":"18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca"}
Sep 04 22:49:41.296322 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:41.296277    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p6g4x" event={"ID":"1c3a1a96-67cc-4068-8bcd-b949c651fd26","Type":"ContainerStarted","Data":"31f3ff3a8854dc20bbe11b432f8983d2940b6996c79977f392c71abf2a643150"}
Sep 04 22:49:41.296402 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:41.296337    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p6g4x" event={"ID":"1c3a1a96-67cc-4068-8bcd-b949c651fd26","Type":"ContainerStarted","Data":"dccc1b8517308454e867d4941064e07bb78885353ec28f936a1c4ad856917844"}
Sep 04 22:49:41.298713 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:41.298688    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw" event={"ID":"e05e4642-b09c-4900-a384-f433d0f6188e","Type":"ContainerStarted","Data":"bd0a08715e7c48c6f2c34195e2264e7678a3850cc4280ed38e387ebff4079204"}
Sep 04 22:49:41.298812 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:41.298745    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw" event={"ID":"e05e4642-b09c-4900-a384-f433d0f6188e","Type":"ContainerStarted","Data":"4beb0f558106ae217c77fc95bece6a4f5da5034c5bed837b8e382ecf988cfd2e"}
Sep 04 22:49:41.336784 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:41.336728    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-p6g4x" podStartSLOduration=2.289606775 podStartE2EDuration="3.336711567s" totalImagesPullingTime="976.440243ms" totalInitContainerRuntime="70.664549ms" isStatefulPod=true podCreationTimestamp="2026-09-04 22:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:41.336166938 +0000 UTC m=+269.446287517" watchObservedRunningTime="2026-09-04 22:49:41.336711567 +0000 UTC m=+269.446832147"
Sep 04 22:49:41.677461 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:41.677375    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:41.777289 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:41.777256    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:42.305138 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:42.305102    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw" event={"ID":"e05e4642-b09c-4900-a384-f433d0f6188e","Type":"ContainerStarted","Data":"82edbd3a725b5c5f1ce32ec1af68176a428574a2628c2f9567cde4ff23add1e0"}
Sep 04 22:49:42.327503 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:42.327449    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2vhtw" podStartSLOduration=2.705348404 podStartE2EDuration="4.327428083s" totalImagesPullingTime="1.622079679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:42.324695053 +0000 UTC m=+270.434815640" watchObservedRunningTime="2026-09-04 22:49:42.327428083 +0000 UTC m=+270.437548673"
Sep 04 22:49:42.964283 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:42.964253    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:42.993758 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:42.993726    2578 patch_prober.go:28] interesting pod/image-registry-77656967dd-bw4qw container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 04 22:49:42.993913 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:42.993776    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-77656967dd-bw4qw" podUID="c98bf6bb-910a-4526-b463-7d6761159691" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 04 22:49:43.026263 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:43.026216    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:43.106652 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:43.106556    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58f67c784b-zrpfj"]
Sep 04 22:49:43.311887 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:43.311847    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerStarted","Data":"0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b"}
Sep 04 22:49:43.312250 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:43.311918    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerStarted","Data":"8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba"}
Sep 04 22:49:43.312250 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:43.311952    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerStarted","Data":"e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5"}
Sep 04 22:49:43.312250 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:43.311982    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerStarted","Data":"7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce"}
Sep 04 22:49:43.312250 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:43.312008    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerStarted","Data":"9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d"}
Sep 04 22:49:43.701713 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:43.701682    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:43.803464 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:43.803428    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:43.964207 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:43.964113    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:44.320447 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:44.320348    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerStarted","Data":"cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc"}
Sep 04 22:49:44.348877 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:44.348812    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.040788919 podStartE2EDuration="5.348789511s" totalImagesPullingTime="3.293621728s" totalInitContainerRuntime="14.378864ms" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:44.348072302 +0000 UTC m=+272.458192890" watchObservedRunningTime="2026-09-04 22:49:44.348789511 +0000 UTC m=+272.458910098"
Sep 04 22:49:44.469789 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:44.469749    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:44.825090 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:44.825045    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:49:48.007436 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.007370    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77656967dd-bw4qw" podUID="c98bf6bb-910a-4526-b463-7d6761159691" containerName="registry" containerID="cri-o://2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38" gracePeriod=30
Sep 04 22:49:48.249725 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.249701    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:49:48.339274 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.339241    2578 generic.go:356] "Generic (PLEG): container finished" podID="c98bf6bb-910a-4526-b463-7d6761159691" containerID="2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38" exitCode=0
Sep 04 22:49:48.339409 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.339280    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77656967dd-bw4qw"
Sep 04 22:49:48.339409 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.339285    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77656967dd-bw4qw" event={"ID":"c98bf6bb-910a-4526-b463-7d6761159691","Type":"ContainerDied","Data":"2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38"}
Sep 04 22:49:48.339409 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.339333    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77656967dd-bw4qw" event={"ID":"c98bf6bb-910a-4526-b463-7d6761159691","Type":"ContainerDied","Data":"01c7ec39f4ce9e12e517b2814b145dc76e857c7f84926a50e051f0f224c6ae9a"}
Sep 04 22:49:48.339409 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.339361    2578 scope.go:117] "RemoveContainer" containerID="2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38"
Sep 04 22:49:48.347065 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.347050    2578 scope.go:117] "RemoveContainer" containerID="2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38"
Sep 04 22:49:48.347320 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:48.347301    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38\": container with ID starting with 2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38 not found: ID does not exist" containerID="2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38"
Sep 04 22:49:48.347384 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.347326    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38"} err="failed to get container status \"2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38\": rpc error: code = NotFound desc = could not find container \"2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38\": container with ID starting with 2d8ed17ffe0a2351f53b60e975fb94e6be693f2de341b8ce5982e87973e6ce38 not found: ID does not exist"
Sep 04 22:49:48.371605 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.371586    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c98bf6bb-910a-4526-b463-7d6761159691-registry-certificates\") pod \"c98bf6bb-910a-4526-b463-7d6761159691\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") "
Sep 04 22:49:48.371670 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.371613    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c98bf6bb-910a-4526-b463-7d6761159691-installation-pull-secrets\") pod \"c98bf6bb-910a-4526-b463-7d6761159691\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") "
Sep 04 22:49:48.371670 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.371632    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-bound-sa-token\") pod \"c98bf6bb-910a-4526-b463-7d6761159691\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") "
Sep 04 22:49:48.371736 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.371677    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c98bf6bb-910a-4526-b463-7d6761159691-ca-trust-extracted\") pod \"c98bf6bb-910a-4526-b463-7d6761159691\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") "
Sep 04 22:49:48.371736 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.371709    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjlg5\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-kube-api-access-tjlg5\") pod \"c98bf6bb-910a-4526-b463-7d6761159691\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") "
Sep 04 22:49:48.371834 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.371760    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c98bf6bb-910a-4526-b463-7d6761159691-trusted-ca\") pod \"c98bf6bb-910a-4526-b463-7d6761159691\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") "
Sep 04 22:49:48.371834 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.371803    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c98bf6bb-910a-4526-b463-7d6761159691-image-registry-private-configuration\") pod \"c98bf6bb-910a-4526-b463-7d6761159691\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") "
Sep 04 22:49:48.371930 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.371849    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls\") pod \"c98bf6bb-910a-4526-b463-7d6761159691\" (UID: \"c98bf6bb-910a-4526-b463-7d6761159691\") "
Sep 04 22:49:48.372042 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.372014    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c98bf6bb-910a-4526-b463-7d6761159691-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c98bf6bb-910a-4526-b463-7d6761159691" (UID: "c98bf6bb-910a-4526-b463-7d6761159691"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:49:48.372146 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.372117    2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c98bf6bb-910a-4526-b463-7d6761159691-registry-certificates\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:49:48.372211 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.372184    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c98bf6bb-910a-4526-b463-7d6761159691-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c98bf6bb-910a-4526-b463-7d6761159691" (UID: "c98bf6bb-910a-4526-b463-7d6761159691"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:49:48.374155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.374089    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c98bf6bb-910a-4526-b463-7d6761159691-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c98bf6bb-910a-4526-b463-7d6761159691" (UID: "c98bf6bb-910a-4526-b463-7d6761159691"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:49:48.374155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.374101    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-kube-api-access-tjlg5" (OuterVolumeSpecName: "kube-api-access-tjlg5") pod "c98bf6bb-910a-4526-b463-7d6761159691" (UID: "c98bf6bb-910a-4526-b463-7d6761159691"). InnerVolumeSpecName "kube-api-access-tjlg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:49:48.374288 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.374185    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c98bf6bb-910a-4526-b463-7d6761159691-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c98bf6bb-910a-4526-b463-7d6761159691" (UID: "c98bf6bb-910a-4526-b463-7d6761159691"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:49:48.374288 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.374190    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c98bf6bb-910a-4526-b463-7d6761159691" (UID: "c98bf6bb-910a-4526-b463-7d6761159691"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:49:48.374356 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.374331    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c98bf6bb-910a-4526-b463-7d6761159691" (UID: "c98bf6bb-910a-4526-b463-7d6761159691"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:49:48.380512 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.380489    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c98bf6bb-910a-4526-b463-7d6761159691-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c98bf6bb-910a-4526-b463-7d6761159691" (UID: "c98bf6bb-910a-4526-b463-7d6761159691"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:49:48.472859 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.472836    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c98bf6bb-910a-4526-b463-7d6761159691-trusted-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:49:48.472859 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.472857    2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c98bf6bb-910a-4526-b463-7d6761159691-image-registry-private-configuration\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:49:48.472976 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.472867    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-registry-tls\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:49:48.472976 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.472876    2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c98bf6bb-910a-4526-b463-7d6761159691-installation-pull-secrets\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:49:48.472976 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.472884    2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-bound-sa-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:49:48.472976 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.472893    2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c98bf6bb-910a-4526-b463-7d6761159691-ca-trust-extracted\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:49:48.472976 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.472901    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjlg5\" (UniqueName: \"kubernetes.io/projected/c98bf6bb-910a-4526-b463-7d6761159691-kube-api-access-tjlg5\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:49:48.655282 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.655213    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77656967dd-bw4qw"]
Sep 04 22:49:48.659320 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.659298    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77656967dd-bw4qw"]
Sep 04 22:49:48.920630 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:48.920555    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:49.021681 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:49.021632    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:50.419629 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:50.419591    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c98bf6bb-910a-4526-b463-7d6761159691" path="/var/lib/kubelet/pods/c98bf6bb-910a-4526-b463-7d6761159691/volumes"
Sep 04 22:49:53.938106 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:49:53.938060    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-mpx5p" podUID="8242b073-28b9-46e8-8783-9aa8871bcf2e"
Sep 04 22:49:54.364484 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:54.364453    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:49:57.134718 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:57.134681    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:49:57.136967 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:57.136947    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8242b073-28b9-46e8-8783-9aa8871bcf2e-metrics-tls\") pod \"dns-default-mpx5p\" (UID: \"8242b073-28b9-46e8-8783-9aa8871bcf2e\") " pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:49:57.367720 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:57.367678    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cv6t6\""
Sep 04 22:49:57.375387 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:57.375350    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:49:57.650367 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:57.650337    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mpx5p"]
Sep 04 22:49:57.652794 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:49:57.652768    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8242b073_28b9_46e8_8783_9aa8871bcf2e.slice/crio-5ce1b0eb1d0661dae03c3469e703bdeae97a10fcf0adbff0e538980f6a5404f5 WatchSource:0}: Error finding container 5ce1b0eb1d0661dae03c3469e703bdeae97a10fcf0adbff0e538980f6a5404f5: Status 404 returned error can't find the container with id 5ce1b0eb1d0661dae03c3469e703bdeae97a10fcf0adbff0e538980f6a5404f5
Sep 04 22:49:58.114754 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:58.114719    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:58.215014 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:58.214975    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:58.385667 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:58.385558    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mpx5p" event={"ID":"8242b073-28b9-46e8-8783-9aa8871bcf2e","Type":"ContainerStarted","Data":"5ce1b0eb1d0661dae03c3469e703bdeae97a10fcf0adbff0e538980f6a5404f5"}
Sep 04 22:49:58.426942 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:58.426912    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:49:59.391531 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:59.391493    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mpx5p" event={"ID":"8242b073-28b9-46e8-8783-9aa8871bcf2e","Type":"ContainerStarted","Data":"ce4d3f2a81596f74c9d32a02df6ce329e1edd26213ff7ec35fb92b00be54f4ee"}
Sep 04 22:49:59.391992 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:59.391552    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mpx5p" event={"ID":"8242b073-28b9-46e8-8783-9aa8871bcf2e","Type":"ContainerStarted","Data":"1c962de97721dbb6b7db69897e75b38c4eb5a15e51b84e1c7b5d04ab0bb23e27"}
Sep 04 22:49:59.391992 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:59.391693    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:49:59.408785 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:49:59.408719    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mpx5p" podStartSLOduration=250.88541912 podStartE2EDuration="4m12.408703766s" totalImagesPullingTime="1.523284646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:45:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:59.407166739 +0000 UTC m=+287.517287328" watchObservedRunningTime="2026-09-04 22:49:59.408703766 +0000 UTC m=+287.518824351"
Sep 04 22:50:07.982937 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:07.982904    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:08.083164 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.083132    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:08.126411 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.126346    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58f67c784b-zrpfj" podUID="8876e514-4ff7-4b06-97f5-b64fea52173f" containerName="console" containerID="cri-o://065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945" gracePeriod=15
Sep 04 22:50:08.183615 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.183588    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:08.239068 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.238987    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:08.342246 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.342213    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:08.680363 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.680335    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:50:08.717652 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.717607    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8876e514-4ff7-4b06-97f5-b64fea52173f-console-serving-cert\") pod \"8876e514-4ff7-4b06-97f5-b64fea52173f\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") "
Sep 04 22:50:08.717652 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.717655    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-service-ca\") pod \"8876e514-4ff7-4b06-97f5-b64fea52173f\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") "
Sep 04 22:50:08.717882 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.717680    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8876e514-4ff7-4b06-97f5-b64fea52173f-console-oauth-config\") pod \"8876e514-4ff7-4b06-97f5-b64fea52173f\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") "
Sep 04 22:50:08.717882 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.717710    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-oauth-serving-cert\") pod \"8876e514-4ff7-4b06-97f5-b64fea52173f\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") "
Sep 04 22:50:08.717882 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.717757    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxjbl\" (UniqueName: \"kubernetes.io/projected/8876e514-4ff7-4b06-97f5-b64fea52173f-kube-api-access-gxjbl\") pod \"8876e514-4ff7-4b06-97f5-b64fea52173f\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") "
Sep 04 22:50:08.718031 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.717894    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-console-config\") pod \"8876e514-4ff7-4b06-97f5-b64fea52173f\" (UID: \"8876e514-4ff7-4b06-97f5-b64fea52173f\") "
Sep 04 22:50:08.718183 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.718158    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-service-ca" (OuterVolumeSpecName: "service-ca") pod "8876e514-4ff7-4b06-97f5-b64fea52173f" (UID: "8876e514-4ff7-4b06-97f5-b64fea52173f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:08.718302 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.718261    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8876e514-4ff7-4b06-97f5-b64fea52173f" (UID: "8876e514-4ff7-4b06-97f5-b64fea52173f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:08.718372 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.718339    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-console-config" (OuterVolumeSpecName: "console-config") pod "8876e514-4ff7-4b06-97f5-b64fea52173f" (UID: "8876e514-4ff7-4b06-97f5-b64fea52173f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:08.719974 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.719950    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8876e514-4ff7-4b06-97f5-b64fea52173f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8876e514-4ff7-4b06-97f5-b64fea52173f" (UID: "8876e514-4ff7-4b06-97f5-b64fea52173f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:08.719974 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.719967    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8876e514-4ff7-4b06-97f5-b64fea52173f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8876e514-4ff7-4b06-97f5-b64fea52173f" (UID: "8876e514-4ff7-4b06-97f5-b64fea52173f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:08.720106 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.720070    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8876e514-4ff7-4b06-97f5-b64fea52173f-kube-api-access-gxjbl" (OuterVolumeSpecName: "kube-api-access-gxjbl") pod "8876e514-4ff7-4b06-97f5-b64fea52173f" (UID: "8876e514-4ff7-4b06-97f5-b64fea52173f"). InnerVolumeSpecName "kube-api-access-gxjbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:50:08.818515 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.818429    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8876e514-4ff7-4b06-97f5-b64fea52173f-console-serving-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:08.818515 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.818458    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:08.818515 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.818472    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8876e514-4ff7-4b06-97f5-b64fea52173f-console-oauth-config\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:08.818515 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.818484    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-oauth-serving-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:08.818515 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.818495    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxjbl\" (UniqueName: \"kubernetes.io/projected/8876e514-4ff7-4b06-97f5-b64fea52173f-kube-api-access-gxjbl\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:08.818515 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:08.818509    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8876e514-4ff7-4b06-97f5-b64fea52173f-console-config\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:09.142084 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:09.142041    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:09.242478 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:09.242445    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:09.398318 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:09.398247    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mpx5p"
Sep 04 22:50:09.430947 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:09.430914    2578 generic.go:356] "Generic (PLEG): container finished" podID="8876e514-4ff7-4b06-97f5-b64fea52173f" containerID="065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945" exitCode=0
Sep 04 22:50:09.431113 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:09.430966    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f67c784b-zrpfj"
Sep 04 22:50:09.431113 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:09.430988    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f67c784b-zrpfj" event={"ID":"8876e514-4ff7-4b06-97f5-b64fea52173f","Type":"ContainerDied","Data":"065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945"}
Sep 04 22:50:09.431113 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:09.431052    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f67c784b-zrpfj" event={"ID":"8876e514-4ff7-4b06-97f5-b64fea52173f","Type":"ContainerDied","Data":"c67bc6ca40ced40685d0b4d45b30397286c202866d0907049ded316b295b421d"}
Sep 04 22:50:09.431250 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:09.431083    2578 scope.go:117] "RemoveContainer" containerID="065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945"
Sep 04 22:50:09.441707 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:09.441684    2578 scope.go:117] "RemoveContainer" containerID="065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945"
Sep 04 22:50:09.441986 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:50:09.441965    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945\": container with ID starting with 065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945 not found: ID does not exist" containerID="065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945"
Sep 04 22:50:09.442077 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:09.441993    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945"} err="failed to get container status \"065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945\": rpc error: code = NotFound desc = could not find container \"065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945\": container with ID starting with 065d668dbfdbb346d43e3833a746c78f17d3d541e60efddd2278849e2cebd945 not found: ID does not exist"
Sep 04 22:50:09.453146 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:09.453119    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58f67c784b-zrpfj"]
Sep 04 22:50:09.456516 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:09.456493    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58f67c784b-zrpfj"]
Sep 04 22:50:10.417656 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:10.417618    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8876e514-4ff7-4b06-97f5-b64fea52173f" path="/var/lib/kubelet/pods/8876e514-4ff7-4b06-97f5-b64fea52173f/volumes"
Sep 04 22:50:10.716095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:10.716024    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mpx5p_8242b073-28b9-46e8-8783-9aa8871bcf2e/dns/0.log"
Sep 04 22:50:10.916170 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:10.916135    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mpx5p_8242b073-28b9-46e8-8783-9aa8871bcf2e/kube-rbac-proxy/0.log"
Sep 04 22:50:11.315797 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:11.315770    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k7nf5_7655d1a4-b050-4eea-b049-7c6899295630/dns-node-resolver/0.log"
Sep 04 22:50:11.650191 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:11.650159    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:11.702004 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:11.701978    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:12.115864 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:12.115841    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4qbrc_407039b1-b5f0-4538-8f92-f277e742b9d6/node-ca/0.log"
Sep 04 22:50:12.300958 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:12.300931    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 22:50:12.302599 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:12.302578    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 22:50:12.314537 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:12.314516    2578 kubelet.go:1648] "Image garbage collection succeeded"
Sep 04 22:50:12.717443 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:12.717416    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67c5f4dcb-cnpgt_fbbca652-c23c-4eaf-b908-7d724be0ab91/router/0.log"
Sep 04 22:50:12.916008 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:12.915980    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hx25v_565701c0-684e-4fc0-9bc5-2ebce20c2100/serve-healthcheck-canary/0.log"
Sep 04 22:50:13.508182 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:13.508148    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:13.571518 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:13.571479    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:15.402762 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:15.402731    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:15.504254 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:15.504221    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:17.379159 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:17.379116    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:17.431696 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:17.431668    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:22.577433 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:22.577399    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:22.677883 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:22.677851    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:23.493843 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:23.493806    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:23.596939 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:23.596900    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:23.697704 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:23.697666    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:24.476809 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:24.476775    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:24.578245 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:24.578209    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:28.500423 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:28.500387    2578 generic.go:356] "Generic (PLEG): container finished" podID="0b1acb3a-ce94-4fa2-963b-f19969e1f344" containerID="5b433ba322715fb0d1204b584b17103966eeadfb446f194507f140b1816ddfea" exitCode=0
Sep 04 22:50:28.500835 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:28.500459    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp" event={"ID":"0b1acb3a-ce94-4fa2-963b-f19969e1f344","Type":"ContainerDied","Data":"5b433ba322715fb0d1204b584b17103966eeadfb446f194507f140b1816ddfea"}
Sep 04 22:50:28.500835 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:28.500774    2578 scope.go:117] "RemoveContainer" containerID="5b433ba322715fb0d1204b584b17103966eeadfb446f194507f140b1816ddfea"
Sep 04 22:50:29.506082 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:29.506046    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9sjbp" event={"ID":"0b1acb3a-ce94-4fa2-963b-f19969e1f344","Type":"ContainerStarted","Data":"916dcd2c7be644c5292ad7175b581daf0189ead5ae07c406287e36fa7e867e5e"}
Sep 04 22:50:30.678973 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:30.678941    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:30.731087 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:30.731052    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:32.385541 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:32.385503    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:32.448429 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:32.448394    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:39.895740 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:39.895706    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:39.996500 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:39.996463    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:40.047424 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:40.047393    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:47.382411 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:47.382378    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:47.435608 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:47.435578    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:51.648657 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:51.648609    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:51.706103 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:51.706069    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:54.345563 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:54.345527    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:54.396676 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:54.396625    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:58.580544 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:58.580495    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 22:50:58.580979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:58.580860    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="alertmanager" containerID="cri-o://9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d" gracePeriod=120
Sep 04 22:50:58.580979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:58.580921    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="kube-rbac-proxy-web" containerID="cri-o://e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5" gracePeriod=120
Sep 04 22:50:58.580979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:58.580932    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="config-reloader" containerID="cri-o://7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce" gracePeriod=120
Sep 04 22:50:58.580979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:58.580949    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="kube-rbac-proxy" containerID="cri-o://8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba" gracePeriod=120
Sep 04 22:50:58.581178 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:58.581002    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="prom-label-proxy" containerID="cri-o://cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc" gracePeriod=120
Sep 04 22:50:58.581178 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:58.580912    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="kube-rbac-proxy-metric" containerID="cri-o://0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b" gracePeriod=120
Sep 04 22:50:59.070208 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.070117    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66fcd75bcd-rds9z"]
Sep 04 22:50:59.070495 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.070478    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8876e514-4ff7-4b06-97f5-b64fea52173f" containerName="console"
Sep 04 22:50:59.070563 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.070497    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8876e514-4ff7-4b06-97f5-b64fea52173f" containerName="console"
Sep 04 22:50:59.070563 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.070513    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c98bf6bb-910a-4526-b463-7d6761159691" containerName="registry"
Sep 04 22:50:59.070563 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.070522    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c98bf6bb-910a-4526-b463-7d6761159691" containerName="registry"
Sep 04 22:50:59.070718 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.070588    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c98bf6bb-910a-4526-b463-7d6761159691" containerName="registry"
Sep 04 22:50:59.070718 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.070603    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8876e514-4ff7-4b06-97f5-b64fea52173f" containerName="console"
Sep 04 22:50:59.073418 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.073404    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.080814 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.080779    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 04 22:50:59.081091 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.081072    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 04 22:50:59.082073 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.082055    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nrl48\""
Sep 04 22:50:59.082228 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.082209    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 04 22:50:59.082330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.082270    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 04 22:50:59.082330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.082298    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 04 22:50:59.083047 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.083027    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 04 22:50:59.083131 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.083111    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 04 22:50:59.089806 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.089783    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 04 22:50:59.098193 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.098161    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66fcd75bcd-rds9z"]
Sep 04 22:50:59.128072 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.128047    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-service-ca\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.128200 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.128083    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-oauth-config\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.128200 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.128149    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-trusted-ca-bundle\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.128293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.128213    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-oauth-serving-cert\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.128293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.128244    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-serving-cert\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.128293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.128285    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpsgn\" (UniqueName: \"kubernetes.io/projected/40ccec3b-b162-4302-8eb9-f80ffbac05c2-kube-api-access-xpsgn\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.128404 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.128313    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-config\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.144033 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.143960    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:59.201732 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.201700    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:50:59.228835 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.228811    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-config\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.228932 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.228856    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-service-ca\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.228932 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.228875    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-oauth-config\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.228932 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.228894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-trusted-ca-bundle\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.229094 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.229072    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-oauth-serving-cert\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.229154 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.229133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-serving-cert\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.229208 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.229173    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpsgn\" (UniqueName: \"kubernetes.io/projected/40ccec3b-b162-4302-8eb9-f80ffbac05c2-kube-api-access-xpsgn\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.229632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.229606    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-service-ca\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.229761 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.229612    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-config\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.229761 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.229722    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-oauth-serving-cert\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.229855 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.229839    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-trusted-ca-bundle\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.231468 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.231445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-oauth-config\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.231549 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.231475    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-serving-cert\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.236790 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.236766    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpsgn\" (UniqueName: \"kubernetes.io/projected/40ccec3b-b162-4302-8eb9-f80ffbac05c2-kube-api-access-xpsgn\") pod \"console-66fcd75bcd-rds9z\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") " pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.383100 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.383066    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:50:59.620672 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.620621    2578 generic.go:356] "Generic (PLEG): container finished" podID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerID="cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc" exitCode=0
Sep 04 22:50:59.620672 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.620661    2578 generic.go:356] "Generic (PLEG): container finished" podID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerID="8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba" exitCode=0
Sep 04 22:50:59.620672 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.620672    2578 generic.go:356] "Generic (PLEG): container finished" podID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerID="7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce" exitCode=0
Sep 04 22:50:59.621069 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.620683    2578 generic.go:356] "Generic (PLEG): container finished" podID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerID="9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d" exitCode=0
Sep 04 22:50:59.621069 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.620679    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerDied","Data":"cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc"}
Sep 04 22:50:59.621069 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.620769    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerDied","Data":"8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba"}
Sep 04 22:50:59.621069 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.620794    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerDied","Data":"7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce"}
Sep 04 22:50:59.621069 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.620817    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerDied","Data":"9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d"}
Sep 04 22:50:59.654104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.654071    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66fcd75bcd-rds9z"]
Sep 04 22:50:59.657907 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:50:59.657885    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40ccec3b_b162_4302_8eb9_f80ffbac05c2.slice/crio-ebcf1bb6771cb2163e95bddcd1fd2681a20ea5d6eb5d806f8114f14584052c32 WatchSource:0}: Error finding container ebcf1bb6771cb2163e95bddcd1fd2681a20ea5d6eb5d806f8114f14584052c32: Status 404 returned error can't find the container with id ebcf1bb6771cb2163e95bddcd1fd2681a20ea5d6eb5d806f8114f14584052c32
Sep 04 22:50:59.815330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.815303    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:50:59.834652 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.834602    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/18c8b332-4fff-4ca8-96f3-3fbd55096106-config-out\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.834808 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.834663    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/18c8b332-4fff-4ca8-96f3-3fbd55096106-tls-assets\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.834808 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.834690    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x98x\" (UniqueName: \"kubernetes.io/projected/18c8b332-4fff-4ca8-96f3-3fbd55096106-kube-api-access-2x98x\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.834808 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.834724    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/18c8b332-4fff-4ca8-96f3-3fbd55096106-alertmanager-main-db\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.834808 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.834762    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c8b332-4fff-4ca8-96f3-3fbd55096106-alertmanager-trusted-ca-bundle\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.834808 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.834805    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-cluster-tls-config\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.835062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.834852    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.835062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.834906    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy-web\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.835062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.834931    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy-metric\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.835062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.834954    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18c8b332-4fff-4ca8-96f3-3fbd55096106-metrics-client-ca\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.835062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.834980    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-main-tls\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.835062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.835013    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-config-volume\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.835062 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.835042    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-web-config\") pod \"18c8b332-4fff-4ca8-96f3-3fbd55096106\" (UID: \"18c8b332-4fff-4ca8-96f3-3fbd55096106\") "
Sep 04 22:50:59.835401 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.835065    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18c8b332-4fff-4ca8-96f3-3fbd55096106-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:50:59.835401 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.835106    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18c8b332-4fff-4ca8-96f3-3fbd55096106-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:59.835401 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.835284    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/18c8b332-4fff-4ca8-96f3-3fbd55096106-alertmanager-main-db\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:59.835401 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.835304    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c8b332-4fff-4ca8-96f3-3fbd55096106-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:59.836079 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.836049    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18c8b332-4fff-4ca8-96f3-3fbd55096106-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:59.837550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.837524    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18c8b332-4fff-4ca8-96f3-3fbd55096106-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:50:59.837715 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.837615    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18c8b332-4fff-4ca8-96f3-3fbd55096106-kube-api-access-2x98x" (OuterVolumeSpecName: "kube-api-access-2x98x") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "kube-api-access-2x98x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:50:59.837824 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.837799    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18c8b332-4fff-4ca8-96f3-3fbd55096106-config-out" (OuterVolumeSpecName: "config-out") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:50:59.838461 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.838429    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:59.838713 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.838675    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:59.839055 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.839008    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:59.839273 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.839247    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-config-volume" (OuterVolumeSpecName: "config-volume") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:59.840290 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.840262    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:59.843764 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.843619    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:59.849874 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.849844    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-web-config" (OuterVolumeSpecName: "web-config") pod "18c8b332-4fff-4ca8-96f3-3fbd55096106" (UID: "18c8b332-4fff-4ca8-96f3-3fbd55096106"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:59.936134 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.936060    2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/18c8b332-4fff-4ca8-96f3-3fbd55096106-tls-assets\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:59.936134 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.936085    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x98x\" (UniqueName: \"kubernetes.io/projected/18c8b332-4fff-4ca8-96f3-3fbd55096106-kube-api-access-2x98x\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:59.936134 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.936095    2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-cluster-tls-config\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:59.936134 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.936106    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:59.936134 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.936116    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:59.936134 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.936124    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:59.936134 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.936132    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18c8b332-4fff-4ca8-96f3-3fbd55096106-metrics-client-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:59.936446 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.936141    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-secret-alertmanager-main-tls\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:59.936446 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.936149    2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:59.936446 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.936157    2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/18c8b332-4fff-4ca8-96f3-3fbd55096106-web-config\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:50:59.936446 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:50:59.936171    2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/18c8b332-4fff-4ca8-96f3-3fbd55096106-config-out\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:51:00.627403 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.627371    2578 generic.go:356] "Generic (PLEG): container finished" podID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerID="0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b" exitCode=0
Sep 04 22:51:00.627403 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.627396    2578 generic.go:356] "Generic (PLEG): container finished" podID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerID="e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5" exitCode=0
Sep 04 22:51:00.627890 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.627454    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.627890 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.627452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerDied","Data":"0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b"}
Sep 04 22:51:00.627890 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.627611    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerDied","Data":"e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5"}
Sep 04 22:51:00.627890 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.627662    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"18c8b332-4fff-4ca8-96f3-3fbd55096106","Type":"ContainerDied","Data":"e943d73cf690a160b419a4491b1f487d018921a674413c36c89fa2f9893389c8"}
Sep 04 22:51:00.627890 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.627673    2578 scope.go:117] "RemoveContainer" containerID="cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc"
Sep 04 22:51:00.629278 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.629247    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66fcd75bcd-rds9z" event={"ID":"40ccec3b-b162-4302-8eb9-f80ffbac05c2","Type":"ContainerStarted","Data":"5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e"}
Sep 04 22:51:00.629380 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.629286    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66fcd75bcd-rds9z" event={"ID":"40ccec3b-b162-4302-8eb9-f80ffbac05c2","Type":"ContainerStarted","Data":"ebcf1bb6771cb2163e95bddcd1fd2681a20ea5d6eb5d806f8114f14584052c32"}
Sep 04 22:51:00.635469 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.635450    2578 scope.go:117] "RemoveContainer" containerID="0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b"
Sep 04 22:51:00.641951 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.641930    2578 scope.go:117] "RemoveContainer" containerID="8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba"
Sep 04 22:51:00.647399 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.647374    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 22:51:00.648819 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.648798    2578 scope.go:117] "RemoveContainer" containerID="e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5"
Sep 04 22:51:00.651982 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.651960    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 22:51:00.656070 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.655940    2578 scope.go:117] "RemoveContainer" containerID="7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce"
Sep 04 22:51:00.662672 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.662655    2578 scope.go:117] "RemoveContainer" containerID="9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d"
Sep 04 22:51:00.669055 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.669008    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66fcd75bcd-rds9z" podStartSLOduration=1.668991075 podStartE2EDuration="1.668991075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:50:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:51:00.667826163 +0000 UTC m=+348.777946749" watchObservedRunningTime="2026-09-04 22:51:00.668991075 +0000 UTC m=+348.779111663"
Sep 04 22:51:00.670091 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.670077    2578 scope.go:117] "RemoveContainer" containerID="18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca"
Sep 04 22:51:00.676814 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.676794    2578 scope.go:117] "RemoveContainer" containerID="cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc"
Sep 04 22:51:00.677070 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:51:00.677053    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc\": container with ID starting with cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc not found: ID does not exist" containerID="cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc"
Sep 04 22:51:00.677118 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.677080    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc"} err="failed to get container status \"cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc\": rpc error: code = NotFound desc = could not find container \"cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc\": container with ID starting with cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc not found: ID does not exist"
Sep 04 22:51:00.677118 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.677096    2578 scope.go:117] "RemoveContainer" containerID="0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b"
Sep 04 22:51:00.677316 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:51:00.677296    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b\": container with ID starting with 0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b not found: ID does not exist" containerID="0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b"
Sep 04 22:51:00.677395 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.677319    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b"} err="failed to get container status \"0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b\": rpc error: code = NotFound desc = could not find container \"0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b\": container with ID starting with 0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b not found: ID does not exist"
Sep 04 22:51:00.677395 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.677336    2578 scope.go:117] "RemoveContainer" containerID="8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba"
Sep 04 22:51:00.677591 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:51:00.677574    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba\": container with ID starting with 8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba not found: ID does not exist" containerID="8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba"
Sep 04 22:51:00.677632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.677596    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba"} err="failed to get container status \"8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba\": rpc error: code = NotFound desc = could not find container \"8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba\": container with ID starting with 8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba not found: ID does not exist"
Sep 04 22:51:00.677632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.677609    2578 scope.go:117] "RemoveContainer" containerID="e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5"
Sep 04 22:51:00.677853 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:51:00.677837    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5\": container with ID starting with e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5 not found: ID does not exist" containerID="e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5"
Sep 04 22:51:00.677893 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.677855    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5"} err="failed to get container status \"e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5\": rpc error: code = NotFound desc = could not find container \"e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5\": container with ID starting with e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5 not found: ID does not exist"
Sep 04 22:51:00.677893 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.677867    2578 scope.go:117] "RemoveContainer" containerID="7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce"
Sep 04 22:51:00.678119 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:51:00.678101    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce\": container with ID starting with 7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce not found: ID does not exist" containerID="7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce"
Sep 04 22:51:00.678168 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.678123    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce"} err="failed to get container status \"7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce\": rpc error: code = NotFound desc = could not find container \"7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce\": container with ID starting with 7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce not found: ID does not exist"
Sep 04 22:51:00.678168 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.678136    2578 scope.go:117] "RemoveContainer" containerID="9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d"
Sep 04 22:51:00.678354 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:51:00.678341    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d\": container with ID starting with 9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d not found: ID does not exist" containerID="9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d"
Sep 04 22:51:00.678398 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.678356    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d"} err="failed to get container status \"9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d\": rpc error: code = NotFound desc = could not find container \"9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d\": container with ID starting with 9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d not found: ID does not exist"
Sep 04 22:51:00.678398 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.678367    2578 scope.go:117] "RemoveContainer" containerID="18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca"
Sep 04 22:51:00.678791 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:51:00.678759    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca\": container with ID starting with 18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca not found: ID does not exist" containerID="18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca"
Sep 04 22:51:00.678952 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.678930    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca"} err="failed to get container status \"18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca\": rpc error: code = NotFound desc = could not find container \"18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca\": container with ID starting with 18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca not found: ID does not exist"
Sep 04 22:51:00.679038 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.678953    2578 scope.go:117] "RemoveContainer" containerID="cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc"
Sep 04 22:51:00.679264 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.679239    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc"} err="failed to get container status \"cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc\": rpc error: code = NotFound desc = could not find container \"cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc\": container with ID starting with cc2d23263c9123986b1eaa708ca0f03f716f17104eec61bc10c45cf2b1ddcdfc not found: ID does not exist"
Sep 04 22:51:00.679337 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.679266    2578 scope.go:117] "RemoveContainer" containerID="0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b"
Sep 04 22:51:00.679555 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.679535    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b"} err="failed to get container status \"0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b\": rpc error: code = NotFound desc = could not find container \"0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b\": container with ID starting with 0da2ee2bee1689bfa958046fd43707b5d633d3d9c5d965f445c842dd4b5b808b not found: ID does not exist"
Sep 04 22:51:00.679631 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.679557    2578 scope.go:117] "RemoveContainer" containerID="8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba"
Sep 04 22:51:00.679863 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.679822    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba"} err="failed to get container status \"8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba\": rpc error: code = NotFound desc = could not find container \"8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba\": container with ID starting with 8c2eb7d149b5e76da6739af26a3885734e7a946843f98fb8f9262761a63960ba not found: ID does not exist"
Sep 04 22:51:00.679863 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.679840    2578 scope.go:117] "RemoveContainer" containerID="e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5"
Sep 04 22:51:00.680068 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680053    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5"} err="failed to get container status \"e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5\": rpc error: code = NotFound desc = could not find container \"e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5\": container with ID starting with e26d4865b3e6b395db2b999d6166a4db44be6bac9d050d13edb5d6a99cc023e5 not found: ID does not exist"
Sep 04 22:51:00.680068 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680068    2578 scope.go:117] "RemoveContainer" containerID="7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce"
Sep 04 22:51:00.680279 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680263    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce"} err="failed to get container status \"7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce\": rpc error: code = NotFound desc = could not find container \"7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce\": container with ID starting with 7a3dfac656a397fb1132413725e6405ebe4cdd1821a89819539b4c51545901ce not found: ID does not exist"
Sep 04 22:51:00.680333 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680278    2578 scope.go:117] "RemoveContainer" containerID="9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d"
Sep 04 22:51:00.680520 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680501    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 22:51:00.680557 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680513    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d"} err="failed to get container status \"9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d\": rpc error: code = NotFound desc = could not find container \"9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d\": container with ID starting with 9f5372bf23094c86b485cf17d13033ead6e58034ddb7d1ad88849e021fcf1c0d not found: ID does not exist"
Sep 04 22:51:00.680557 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680532    2578 scope.go:117] "RemoveContainer" containerID="18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca"
Sep 04 22:51:00.680788 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680767    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca"} err="failed to get container status \"18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca\": rpc error: code = NotFound desc = could not find container \"18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca\": container with ID starting with 18acfa31bc97363e94cc19147582126d56c6ecacda2d5bab18dda741060438ca not found: ID does not exist"
Sep 04 22:51:00.680839 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680808    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="init-config-reloader"
Sep 04 22:51:00.680839 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680820    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="init-config-reloader"
Sep 04 22:51:00.680839 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680829    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="alertmanager"
Sep 04 22:51:00.680839 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680835    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="alertmanager"
Sep 04 22:51:00.680949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680841    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="kube-rbac-proxy-web"
Sep 04 22:51:00.680949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680847    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="kube-rbac-proxy-web"
Sep 04 22:51:00.680949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680859    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="kube-rbac-proxy"
Sep 04 22:51:00.680949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680864    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="kube-rbac-proxy"
Sep 04 22:51:00.680949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680870    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="prom-label-proxy"
Sep 04 22:51:00.680949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680876    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="prom-label-proxy"
Sep 04 22:51:00.680949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680883    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="config-reloader"
Sep 04 22:51:00.680949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680888    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="config-reloader"
Sep 04 22:51:00.680949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680893    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="kube-rbac-proxy-metric"
Sep 04 22:51:00.680949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680898    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="kube-rbac-proxy-metric"
Sep 04 22:51:00.680949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680943    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="config-reloader"
Sep 04 22:51:00.680949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680949    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="kube-rbac-proxy-metric"
Sep 04 22:51:00.681309 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680955    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="prom-label-proxy"
Sep 04 22:51:00.681309 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680963    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="alertmanager"
Sep 04 22:51:00.681309 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680971    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="kube-rbac-proxy-web"
Sep 04 22:51:00.681309 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.680976    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" containerName="kube-rbac-proxy"
Sep 04 22:51:00.685824 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.685807    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.688139 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.688121    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 04 22:51:00.688250 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.688123    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 04 22:51:00.688250 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.688227    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 04 22:51:00.688588 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.688463    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jbw85\""
Sep 04 22:51:00.688588 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.688470    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 04 22:51:00.688588 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.688497    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 04 22:51:00.688588 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.688515    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 04 22:51:00.688588 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.688524    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 04 22:51:00.688588 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.688472    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 04 22:51:00.695753 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.695726    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 04 22:51:00.700399 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.700378    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 22:51:00.742326 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.742470 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d141747-26de-41f3-80b8-1544d1bc444b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.742470 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742390    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqtc8\" (UniqueName: \"kubernetes.io/projected/5d141747-26de-41f3-80b8-1544d1bc444b-kube-api-access-rqtc8\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.742470 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742417    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d141747-26de-41f3-80b8-1544d1bc444b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.742585 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742468    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-web-config\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.742676 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.742728 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742689    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d141747-26de-41f3-80b8-1544d1bc444b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.742728 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742709    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d141747-26de-41f3-80b8-1544d1bc444b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.742728 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742724    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d141747-26de-41f3-80b8-1544d1bc444b-config-out\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.742814 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742743    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.742881 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-config-volume\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.742938 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742927    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.742973 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.742955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.843860 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.843830    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.843971 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.843868    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.843971 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.843902    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d141747-26de-41f3-80b8-1544d1bc444b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.843971 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.843925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqtc8\" (UniqueName: \"kubernetes.io/projected/5d141747-26de-41f3-80b8-1544d1bc444b-kube-api-access-rqtc8\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.843971 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.843951    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d141747-26de-41f3-80b8-1544d1bc444b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.844142 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.843976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-web-config\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.844142 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.844023    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.844142 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.844045    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d141747-26de-41f3-80b8-1544d1bc444b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.844142 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.844066    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d141747-26de-41f3-80b8-1544d1bc444b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.844142 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.844104    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d141747-26de-41f3-80b8-1544d1bc444b-config-out\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.844306 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.844147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.844306 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.844182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-config-volume\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.844306 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.844214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.844718 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.844690    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d141747-26de-41f3-80b8-1544d1bc444b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.845289 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.845266    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d141747-26de-41f3-80b8-1544d1bc444b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.845411 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.845323    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d141747-26de-41f3-80b8-1544d1bc444b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.847134 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.847040    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.847134 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.847071    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.847293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.847177    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-web-config\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.847293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.847203    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.847293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.847281    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.847810 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.847785    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-config-volume\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.847810 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.847799    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d141747-26de-41f3-80b8-1544d1bc444b-config-out\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.848007 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.847991    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d141747-26de-41f3-80b8-1544d1bc444b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.848947 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.848931    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d141747-26de-41f3-80b8-1544d1bc444b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.852230 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.852211    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqtc8\" (UniqueName: \"kubernetes.io/projected/5d141747-26de-41f3-80b8-1544d1bc444b-kube-api-access-rqtc8\") pod \"alertmanager-main-0\" (UID: \"5d141747-26de-41f3-80b8-1544d1bc444b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:00.997003 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:00.996929    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 04 22:51:01.281381 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:01.281216    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 22:51:01.283285 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:51:01.283256    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d141747_26de_41f3_80b8_1544d1bc444b.slice/crio-cc81709d4692145d19a442628e557cf61fd603d0e826f60ae3afeb233e6bb6c1 WatchSource:0}: Error finding container cc81709d4692145d19a442628e557cf61fd603d0e826f60ae3afeb233e6bb6c1: Status 404 returned error can't find the container with id cc81709d4692145d19a442628e557cf61fd603d0e826f60ae3afeb233e6bb6c1
Sep 04 22:51:01.634564 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:01.634531    2578 generic.go:356] "Generic (PLEG): container finished" podID="5d141747-26de-41f3-80b8-1544d1bc444b" containerID="a5b1352d55e453dac28ded2a8656bf8c813f213a3c2c1c9590ce96a7639acb48" exitCode=0
Sep 04 22:51:01.634981 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:01.634625    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d141747-26de-41f3-80b8-1544d1bc444b","Type":"ContainerDied","Data":"a5b1352d55e453dac28ded2a8656bf8c813f213a3c2c1c9590ce96a7639acb48"}
Sep 04 22:51:01.634981 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:01.634713    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d141747-26de-41f3-80b8-1544d1bc444b","Type":"ContainerStarted","Data":"cc81709d4692145d19a442628e557cf61fd603d0e826f60ae3afeb233e6bb6c1"}
Sep 04 22:51:02.419694 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:02.419663    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18c8b332-4fff-4ca8-96f3-3fbd55096106" path="/var/lib/kubelet/pods/18c8b332-4fff-4ca8-96f3-3fbd55096106/volumes"
Sep 04 22:51:02.644627 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:02.644585    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d141747-26de-41f3-80b8-1544d1bc444b","Type":"ContainerStarted","Data":"b268d086c3d6b1e3da8ec89dff6303e7350aeb59cef9312ea673f7eff8089f0b"}
Sep 04 22:51:02.645021 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:02.644688    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d141747-26de-41f3-80b8-1544d1bc444b","Type":"ContainerStarted","Data":"819d811415203f2b164a8f927b9b40e699ed8e82406ff75e5d758c1158fb0c61"}
Sep 04 22:51:02.645021 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:02.644736    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d141747-26de-41f3-80b8-1544d1bc444b","Type":"ContainerStarted","Data":"8906d1adc01f11d981bf7610e731cc032796876073a4fdb6027126c6e65c4214"}
Sep 04 22:51:02.645021 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:02.644778    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d141747-26de-41f3-80b8-1544d1bc444b","Type":"ContainerStarted","Data":"dfd9303e5939652a503c086831a7de02f4114ca4db512fb039c0beaa245a9713"}
Sep 04 22:51:02.645021 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:02.644816    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d141747-26de-41f3-80b8-1544d1bc444b","Type":"ContainerStarted","Data":"c3a5ca99db7117699d41ead45e9b56a2a4b42564ab57891b50f4ed29b68adb9e"}
Sep 04 22:51:02.645021 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:02.644860    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d141747-26de-41f3-80b8-1544d1bc444b","Type":"ContainerStarted","Data":"33225252d15a8232be9bfb5095d7c5d9c8e1f5c02d3775f762eb631d93e692b2"}
Sep 04 22:51:02.680053 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:02.677777    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.6611279359999997 podStartE2EDuration="2.677749723s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.621787ms" isStatefulPod=false podCreationTimestamp="2026-09-04 22:51:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:51:02.67201913 +0000 UTC m=+350.782139716" watchObservedRunningTime="2026-09-04 22:51:02.677749723 +0000 UTC m=+350.787870309"
Sep 04 22:51:06.523582 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:06.523497    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:51:06.625426 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:06.625383    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:51:09.383880 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:09.383840    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:51:09.384254 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:09.383907    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:51:09.388696 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:09.388668    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:51:09.675562 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:09.675485    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:51:09.855371 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:09.855328    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:51:09.950186 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:09.950093    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:51:09.999862 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:09.999834    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:51:17.400463 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:17.400403    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:51:17.494861 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:17.494823    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:51:39.411832 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:39.411789    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:51:39.467146 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:39.467111    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:51:56.270897 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:56.270863    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:51:56.373594 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:56.373558    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:51:56.915513 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:56.915476    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"]
Sep 04 22:51:56.918949 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:56.918932    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:51:56.921595 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:56.921563    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 04 22:51:56.921595 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:56.921574    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 04 22:51:56.921789 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:56.921574    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nz8jb\""
Sep 04 22:51:56.927324 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:56.927298    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"]
Sep 04 22:51:56.982995 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:56.982955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26e6c198-3511-49c2-ac74-74b0624778d4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6\" (UID: \"26e6c198-3511-49c2-ac74-74b0624778d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:51:56.983173 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:56.983006    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5zds\" (UniqueName: \"kubernetes.io/projected/26e6c198-3511-49c2-ac74-74b0624778d4-kube-api-access-r5zds\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6\" (UID: \"26e6c198-3511-49c2-ac74-74b0624778d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:51:56.983173 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:56.983034    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26e6c198-3511-49c2-ac74-74b0624778d4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6\" (UID: \"26e6c198-3511-49c2-ac74-74b0624778d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:51:57.083863 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:57.083830    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5zds\" (UniqueName: \"kubernetes.io/projected/26e6c198-3511-49c2-ac74-74b0624778d4-kube-api-access-r5zds\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6\" (UID: \"26e6c198-3511-49c2-ac74-74b0624778d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:51:57.084013 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:57.083880    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26e6c198-3511-49c2-ac74-74b0624778d4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6\" (UID: \"26e6c198-3511-49c2-ac74-74b0624778d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:51:57.084013 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:57.083926    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26e6c198-3511-49c2-ac74-74b0624778d4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6\" (UID: \"26e6c198-3511-49c2-ac74-74b0624778d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:51:57.084260 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:57.084243    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26e6c198-3511-49c2-ac74-74b0624778d4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6\" (UID: \"26e6c198-3511-49c2-ac74-74b0624778d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:51:57.084330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:57.084280    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26e6c198-3511-49c2-ac74-74b0624778d4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6\" (UID: \"26e6c198-3511-49c2-ac74-74b0624778d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:51:57.092999 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:57.092974    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5zds\" (UniqueName: \"kubernetes.io/projected/26e6c198-3511-49c2-ac74-74b0624778d4-kube-api-access-r5zds\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6\" (UID: \"26e6c198-3511-49c2-ac74-74b0624778d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:51:57.228842 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:57.228761    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:51:57.496598 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:57.496574    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"]
Sep 04 22:51:57.499160 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:51:57.499132    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e6c198_3511_49c2_ac74_74b0624778d4.slice/crio-5e2fba25d5fecfd09f61e7f8dff192ba152eb1d67bdddf39f90a4fbe2f3109c1 WatchSource:0}: Error finding container 5e2fba25d5fecfd09f61e7f8dff192ba152eb1d67bdddf39f90a4fbe2f3109c1: Status 404 returned error can't find the container with id 5e2fba25d5fecfd09f61e7f8dff192ba152eb1d67bdddf39f90a4fbe2f3109c1
Sep 04 22:51:57.501184 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:57.501168    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 22:51:57.844099 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:51:57.844066    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6" event={"ID":"26e6c198-3511-49c2-ac74-74b0624778d4","Type":"ContainerStarted","Data":"5e2fba25d5fecfd09f61e7f8dff192ba152eb1d67bdddf39f90a4fbe2f3109c1"}
Sep 04 22:52:03.870876 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:03.870840    2578 generic.go:356] "Generic (PLEG): container finished" podID="26e6c198-3511-49c2-ac74-74b0624778d4" containerID="01d70a81ec34a05871e587acf39d2314d461835542f6b20c61233fb9e0970d13" exitCode=0
Sep 04 22:52:03.871296 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:03.870925    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6" event={"ID":"26e6c198-3511-49c2-ac74-74b0624778d4","Type":"ContainerDied","Data":"01d70a81ec34a05871e587acf39d2314d461835542f6b20c61233fb9e0970d13"}
Sep 04 22:52:05.881763 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:05.881727    2578 generic.go:356] "Generic (PLEG): container finished" podID="26e6c198-3511-49c2-ac74-74b0624778d4" containerID="11876cc5e588931b1d5a58d59aa518bd478538004e9cf2b55091a5a7255de2de" exitCode=0
Sep 04 22:52:05.882092 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:05.881777    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6" event={"ID":"26e6c198-3511-49c2-ac74-74b0624778d4","Type":"ContainerDied","Data":"11876cc5e588931b1d5a58d59aa518bd478538004e9cf2b55091a5a7255de2de"}
Sep 04 22:52:12.916555 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:12.916516    2578 generic.go:356] "Generic (PLEG): container finished" podID="26e6c198-3511-49c2-ac74-74b0624778d4" containerID="e9aa18d362c8e14a6892d1df72f34a9706a44ba5711241f3aac469119859f777" exitCode=0
Sep 04 22:52:12.916938 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:12.916601    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6" event={"ID":"26e6c198-3511-49c2-ac74-74b0624778d4","Type":"ContainerDied","Data":"e9aa18d362c8e14a6892d1df72f34a9706a44ba5711241f3aac469119859f777"}
Sep 04 22:52:14.044725 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.044698    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:52:14.130104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.130065    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26e6c198-3511-49c2-ac74-74b0624778d4-util\") pod \"26e6c198-3511-49c2-ac74-74b0624778d4\" (UID: \"26e6c198-3511-49c2-ac74-74b0624778d4\") "
Sep 04 22:52:14.130276 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.130119    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26e6c198-3511-49c2-ac74-74b0624778d4-bundle\") pod \"26e6c198-3511-49c2-ac74-74b0624778d4\" (UID: \"26e6c198-3511-49c2-ac74-74b0624778d4\") "
Sep 04 22:52:14.130276 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.130159    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5zds\" (UniqueName: \"kubernetes.io/projected/26e6c198-3511-49c2-ac74-74b0624778d4-kube-api-access-r5zds\") pod \"26e6c198-3511-49c2-ac74-74b0624778d4\" (UID: \"26e6c198-3511-49c2-ac74-74b0624778d4\") "
Sep 04 22:52:14.130837 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.130810    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e6c198-3511-49c2-ac74-74b0624778d4-bundle" (OuterVolumeSpecName: "bundle") pod "26e6c198-3511-49c2-ac74-74b0624778d4" (UID: "26e6c198-3511-49c2-ac74-74b0624778d4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:52:14.132391 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.132368    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26e6c198-3511-49c2-ac74-74b0624778d4-kube-api-access-r5zds" (OuterVolumeSpecName: "kube-api-access-r5zds") pod "26e6c198-3511-49c2-ac74-74b0624778d4" (UID: "26e6c198-3511-49c2-ac74-74b0624778d4"). InnerVolumeSpecName "kube-api-access-r5zds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:52:14.134137 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.134116    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e6c198-3511-49c2-ac74-74b0624778d4-util" (OuterVolumeSpecName: "util") pod "26e6c198-3511-49c2-ac74-74b0624778d4" (UID: "26e6c198-3511-49c2-ac74-74b0624778d4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:52:14.231295 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.231209    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26e6c198-3511-49c2-ac74-74b0624778d4-util\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:52:14.231295 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.231239    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26e6c198-3511-49c2-ac74-74b0624778d4-bundle\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:52:14.231295 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.231249    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5zds\" (UniqueName: \"kubernetes.io/projected/26e6c198-3511-49c2-ac74-74b0624778d4-kube-api-access-r5zds\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:52:14.926255 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.926224    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6"
Sep 04 22:52:14.926255 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.926234    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khcc6" event={"ID":"26e6c198-3511-49c2-ac74-74b0624778d4","Type":"ContainerDied","Data":"5e2fba25d5fecfd09f61e7f8dff192ba152eb1d67bdddf39f90a4fbe2f3109c1"}
Sep 04 22:52:14.926459 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:14.926282    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e2fba25d5fecfd09f61e7f8dff192ba152eb1d67bdddf39f90a4fbe2f3109c1"
Sep 04 22:52:17.892220 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:17.892189    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:18.688891 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:18.688859    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:19.845359 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.845324    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6"]
Sep 04 22:52:19.845809 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.845626    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26e6c198-3511-49c2-ac74-74b0624778d4" containerName="util"
Sep 04 22:52:19.845809 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.845654    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="26e6c198-3511-49c2-ac74-74b0624778d4" containerName="util"
Sep 04 22:52:19.845809 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.845663    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26e6c198-3511-49c2-ac74-74b0624778d4" containerName="pull"
Sep 04 22:52:19.845809 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.845669    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="26e6c198-3511-49c2-ac74-74b0624778d4" containerName="pull"
Sep 04 22:52:19.845809 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.845675    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26e6c198-3511-49c2-ac74-74b0624778d4" containerName="extract"
Sep 04 22:52:19.845809 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.845682    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="26e6c198-3511-49c2-ac74-74b0624778d4" containerName="extract"
Sep 04 22:52:19.845809 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.845732    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="26e6c198-3511-49c2-ac74-74b0624778d4" containerName="extract"
Sep 04 22:52:19.889597 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.889566    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6"]
Sep 04 22:52:19.889767 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.889685    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6"
Sep 04 22:52:19.892690 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.892663    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 04 22:52:19.892859 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.892839    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-4r2jb\""
Sep 04 22:52:19.893072 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.893058    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 04 22:52:19.979892 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.979854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98033a46-c710-46a0-bc22-4a52831e55bc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mk2l6\" (UID: \"98033a46-c710-46a0-bc22-4a52831e55bc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6"
Sep 04 22:52:19.980057 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:19.979934    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqm8k\" (UniqueName: \"kubernetes.io/projected/98033a46-c710-46a0-bc22-4a52831e55bc-kube-api-access-qqm8k\") pod \"cert-manager-operator-controller-manager-55c444bf86-mk2l6\" (UID: \"98033a46-c710-46a0-bc22-4a52831e55bc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6"
Sep 04 22:52:20.081182 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:20.081148    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98033a46-c710-46a0-bc22-4a52831e55bc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mk2l6\" (UID: \"98033a46-c710-46a0-bc22-4a52831e55bc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6"
Sep 04 22:52:20.081335 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:20.081211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqm8k\" (UniqueName: \"kubernetes.io/projected/98033a46-c710-46a0-bc22-4a52831e55bc-kube-api-access-qqm8k\") pod \"cert-manager-operator-controller-manager-55c444bf86-mk2l6\" (UID: \"98033a46-c710-46a0-bc22-4a52831e55bc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6"
Sep 04 22:52:20.081545 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:20.081526    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98033a46-c710-46a0-bc22-4a52831e55bc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mk2l6\" (UID: \"98033a46-c710-46a0-bc22-4a52831e55bc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6"
Sep 04 22:52:20.090565 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:20.090543    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqm8k\" (UniqueName: \"kubernetes.io/projected/98033a46-c710-46a0-bc22-4a52831e55bc-kube-api-access-qqm8k\") pod \"cert-manager-operator-controller-manager-55c444bf86-mk2l6\" (UID: \"98033a46-c710-46a0-bc22-4a52831e55bc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6"
Sep 04 22:52:20.198383 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:20.198300    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6"
Sep 04 22:52:20.504744 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:20.504656    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6"]
Sep 04 22:52:20.507794 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:52:20.507758    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98033a46_c710_46a0_bc22_4a52831e55bc.slice/crio-f1d5782d6337cafc274b4036ff1fc9b1442128ea068452c42f41fc1b889d549c WatchSource:0}: Error finding container f1d5782d6337cafc274b4036ff1fc9b1442128ea068452c42f41fc1b889d549c: Status 404 returned error can't find the container with id f1d5782d6337cafc274b4036ff1fc9b1442128ea068452c42f41fc1b889d549c
Sep 04 22:52:20.951181 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:20.951145    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6" event={"ID":"98033a46-c710-46a0-bc22-4a52831e55bc","Type":"ContainerStarted","Data":"f1d5782d6337cafc274b4036ff1fc9b1442128ea068452c42f41fc1b889d549c"}
Sep 04 22:52:21.522012 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:21.521971    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:21.621827 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:21.621790    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:23.613244 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:23.613213    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:23.711299 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:23.711268    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:23.964790 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:23.964709    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6" event={"ID":"98033a46-c710-46a0-bc22-4a52831e55bc","Type":"ContainerStarted","Data":"8b51cf4a214740260173c4157b9472469d745bb6c89ed8444dc7a37af14b2813"}
Sep 04 22:52:23.986249 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:23.986188    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mk2l6" podStartSLOduration=2.569091779 podStartE2EDuration="4.986168565s" totalImagesPullingTime="2.417076786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:52:23.98404092 +0000 UTC m=+432.094161506" watchObservedRunningTime="2026-09-04 22:52:23.986168565 +0000 UTC m=+432.096289152"
Sep 04 22:52:24.423883 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.423851    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:24.524797 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.524762    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:24.545575 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.545543    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"]
Sep 04 22:52:24.562693 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.562654    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"]
Sep 04 22:52:24.562858 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.562738    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:24.565625 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.565600    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nz8jb\""
Sep 04 22:52:24.565776 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.565606    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 04 22:52:24.565776 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.565672    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 04 22:52:24.619295 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.619260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk\" (UID: \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:24.619698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.619303    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk\" (UID: \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:24.619698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.619330    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmnzm\" (UniqueName: \"kubernetes.io/projected/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-kube-api-access-tmnzm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk\" (UID: \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:24.720312 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.720224    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk\" (UID: \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:24.720312 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.720276    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk\" (UID: \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:24.720494 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.720319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmnzm\" (UniqueName: \"kubernetes.io/projected/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-kube-api-access-tmnzm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk\" (UID: \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:24.720727 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.720697    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk\" (UID: \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:24.720831 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.720703    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk\" (UID: \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:24.742738 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.742703    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmnzm\" (UniqueName: \"kubernetes.io/projected/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-kube-api-access-tmnzm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk\" (UID: \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:24.872780 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:24.872738    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:25.158854 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:25.158822    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"]
Sep 04 22:52:25.162357 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:52:25.162330    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0115d0cc_18fa_400e_92b6_d1df77e7d1f5.slice/crio-f69a321d716525004453053433472b2aebed486c5eaf3c982995d1ff9d38ce5f WatchSource:0}: Error finding container f69a321d716525004453053433472b2aebed486c5eaf3c982995d1ff9d38ce5f: Status 404 returned error can't find the container with id f69a321d716525004453053433472b2aebed486c5eaf3c982995d1ff9d38ce5f
Sep 04 22:52:25.627749 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:25.627722    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:25.975186 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:25.975090    2578 generic.go:356] "Generic (PLEG): container finished" podID="0115d0cc-18fa-400e-92b6-d1df77e7d1f5" containerID="4e3b9c8e658e532bad0c872548403d177e1592a57344828464ea522cd2e4e8b1" exitCode=0
Sep 04 22:52:25.975321 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:25.975173    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk" event={"ID":"0115d0cc-18fa-400e-92b6-d1df77e7d1f5","Type":"ContainerDied","Data":"4e3b9c8e658e532bad0c872548403d177e1592a57344828464ea522cd2e4e8b1"}
Sep 04 22:52:25.975321 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:25.975223    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk" event={"ID":"0115d0cc-18fa-400e-92b6-d1df77e7d1f5","Type":"ContainerStarted","Data":"f69a321d716525004453053433472b2aebed486c5eaf3c982995d1ff9d38ce5f"}
Sep 04 22:52:26.013972 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.013944    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:26.251724 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.251653    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm"]
Sep 04 22:52:26.253927 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.253909    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm"
Sep 04 22:52:26.256743 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.256722    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-txp7n\""
Sep 04 22:52:26.256861 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.256761    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 04 22:52:26.256861 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.256770    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 04 22:52:26.265675 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.265653    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm"]
Sep 04 22:52:26.332421 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.332391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgb7j\" (UniqueName: \"kubernetes.io/projected/3cd218ed-da94-4933-be24-8046cc1c5dcb-kube-api-access-hgb7j\") pod \"cert-manager-cainjector-559cdd8c87-q4fpm\" (UID: \"3cd218ed-da94-4933-be24-8046cc1c5dcb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm"
Sep 04 22:52:26.332421 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.332424    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3cd218ed-da94-4933-be24-8046cc1c5dcb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-q4fpm\" (UID: \"3cd218ed-da94-4933-be24-8046cc1c5dcb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm"
Sep 04 22:52:26.433340 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.433293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgb7j\" (UniqueName: \"kubernetes.io/projected/3cd218ed-da94-4933-be24-8046cc1c5dcb-kube-api-access-hgb7j\") pod \"cert-manager-cainjector-559cdd8c87-q4fpm\" (UID: \"3cd218ed-da94-4933-be24-8046cc1c5dcb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm"
Sep 04 22:52:26.433340 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.433344    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3cd218ed-da94-4933-be24-8046cc1c5dcb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-q4fpm\" (UID: \"3cd218ed-da94-4933-be24-8046cc1c5dcb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm"
Sep 04 22:52:26.441451 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.441410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3cd218ed-da94-4933-be24-8046cc1c5dcb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-q4fpm\" (UID: \"3cd218ed-da94-4933-be24-8046cc1c5dcb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm"
Sep 04 22:52:26.441682 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.441635    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgb7j\" (UniqueName: \"kubernetes.io/projected/3cd218ed-da94-4933-be24-8046cc1c5dcb-kube-api-access-hgb7j\") pod \"cert-manager-cainjector-559cdd8c87-q4fpm\" (UID: \"3cd218ed-da94-4933-be24-8046cc1c5dcb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm"
Sep 04 22:52:26.562296 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.562210    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm"
Sep 04 22:52:26.840726 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.840692    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm"]
Sep 04 22:52:26.842425 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:52:26.842399    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3cd218ed_da94_4933_be24_8046cc1c5dcb.slice/crio-38fb103d940abf5f1c7812a0487f2e9ce1485ec1f4545ba1136b2114c8d725a6 WatchSource:0}: Error finding container 38fb103d940abf5f1c7812a0487f2e9ce1485ec1f4545ba1136b2114c8d725a6: Status 404 returned error can't find the container with id 38fb103d940abf5f1c7812a0487f2e9ce1485ec1f4545ba1136b2114c8d725a6
Sep 04 22:52:26.982309 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:26.982273    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm" event={"ID":"3cd218ed-da94-4933-be24-8046cc1c5dcb","Type":"ContainerStarted","Data":"38fb103d940abf5f1c7812a0487f2e9ce1485ec1f4545ba1136b2114c8d725a6"}
Sep 04 22:52:27.989065 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:27.989030    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk" event={"ID":"0115d0cc-18fa-400e-92b6-d1df77e7d1f5","Type":"ContainerStarted","Data":"19d5dd9e3b1f31fb9d89a08a11229c775ddf77a5b5572ade7b3571d4b7153535"}
Sep 04 22:52:28.309961 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:28.309932    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:28.410518 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:28.410436    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:28.995985 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:28.995943    2578 generic.go:356] "Generic (PLEG): container finished" podID="0115d0cc-18fa-400e-92b6-d1df77e7d1f5" containerID="19d5dd9e3b1f31fb9d89a08a11229c775ddf77a5b5572ade7b3571d4b7153535" exitCode=0
Sep 04 22:52:28.995985 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:28.995966    2578 generic.go:356] "Generic (PLEG): container finished" podID="0115d0cc-18fa-400e-92b6-d1df77e7d1f5" containerID="3a87e9796862c04ed64ced44057823089f879a49bc2c54dd1bb680120b135ed1" exitCode=0
Sep 04 22:52:28.996409 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:28.996008    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk" event={"ID":"0115d0cc-18fa-400e-92b6-d1df77e7d1f5","Type":"ContainerDied","Data":"19d5dd9e3b1f31fb9d89a08a11229c775ddf77a5b5572ade7b3571d4b7153535"}
Sep 04 22:52:28.996409 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:28.996074    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk" event={"ID":"0115d0cc-18fa-400e-92b6-d1df77e7d1f5","Type":"ContainerDied","Data":"3a87e9796862c04ed64ced44057823089f879a49bc2c54dd1bb680120b135ed1"}
Sep 04 22:52:30.002477 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:30.002431    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm" event={"ID":"3cd218ed-da94-4933-be24-8046cc1c5dcb","Type":"ContainerStarted","Data":"19ea897c3fef8ad67489898091a26ca181d24e6d7a99205fe5c612eed96703d8"}
Sep 04 22:52:30.128843 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:30.128818    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:30.145751 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:30.145552    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q4fpm" podStartSLOduration=1.131036318 podStartE2EDuration="4.145531555s" totalImagesPullingTime="3.014495237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:52:30.017876286 +0000 UTC m=+438.127996872" watchObservedRunningTime="2026-09-04 22:52:30.145531555 +0000 UTC m=+438.255652142"
Sep 04 22:52:30.268287 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:30.268245    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-util\") pod \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\" (UID: \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\") "
Sep 04 22:52:30.268477 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:30.268317    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-bundle\") pod \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\" (UID: \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\") "
Sep 04 22:52:30.268477 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:30.268363    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmnzm\" (UniqueName: \"kubernetes.io/projected/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-kube-api-access-tmnzm\") pod \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\" (UID: \"0115d0cc-18fa-400e-92b6-d1df77e7d1f5\") "
Sep 04 22:52:30.268706 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:30.268676    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-bundle" (OuterVolumeSpecName: "bundle") pod "0115d0cc-18fa-400e-92b6-d1df77e7d1f5" (UID: "0115d0cc-18fa-400e-92b6-d1df77e7d1f5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:52:30.270437 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:30.270410    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-kube-api-access-tmnzm" (OuterVolumeSpecName: "kube-api-access-tmnzm") pod "0115d0cc-18fa-400e-92b6-d1df77e7d1f5" (UID: "0115d0cc-18fa-400e-92b6-d1df77e7d1f5"). InnerVolumeSpecName "kube-api-access-tmnzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:52:30.273200 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:30.273175    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-util" (OuterVolumeSpecName: "util") pod "0115d0cc-18fa-400e-92b6-d1df77e7d1f5" (UID: "0115d0cc-18fa-400e-92b6-d1df77e7d1f5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:52:30.369228 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:30.369154    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-util\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:52:30.369228 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:30.369180    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-bundle\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:52:30.369228 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:30.369191    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmnzm\" (UniqueName: \"kubernetes.io/projected/0115d0cc-18fa-400e-92b6-d1df77e7d1f5-kube-api-access-tmnzm\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:52:31.008818 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.008788    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk"
Sep 04 22:52:31.009257 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.008822    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f694jk" event={"ID":"0115d0cc-18fa-400e-92b6-d1df77e7d1f5","Type":"ContainerDied","Data":"f69a321d716525004453053433472b2aebed486c5eaf3c982995d1ff9d38ce5f"}
Sep 04 22:52:31.009257 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.008873    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f69a321d716525004453053433472b2aebed486c5eaf3c982995d1ff9d38ce5f"
Sep 04 22:52:31.610593 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.610553    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mkp8g"]
Sep 04 22:52:31.610940 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.610922    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0115d0cc-18fa-400e-92b6-d1df77e7d1f5" containerName="util"
Sep 04 22:52:31.611028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.610942    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0115d0cc-18fa-400e-92b6-d1df77e7d1f5" containerName="util"
Sep 04 22:52:31.611028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.610970    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0115d0cc-18fa-400e-92b6-d1df77e7d1f5" containerName="pull"
Sep 04 22:52:31.611028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.610979    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0115d0cc-18fa-400e-92b6-d1df77e7d1f5" containerName="pull"
Sep 04 22:52:31.611028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.610990    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0115d0cc-18fa-400e-92b6-d1df77e7d1f5" containerName="extract"
Sep 04 22:52:31.611028 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.611001    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0115d0cc-18fa-400e-92b6-d1df77e7d1f5" containerName="extract"
Sep 04 22:52:31.611255 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.611107    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0115d0cc-18fa-400e-92b6-d1df77e7d1f5" containerName="extract"
Sep 04 22:52:31.612936 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.612913    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g"
Sep 04 22:52:31.615664 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.615622    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-l7rrc\""
Sep 04 22:52:31.626627 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.626602    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mkp8g"]
Sep 04 22:52:31.779163 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.779130    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1af8bc5e-c050-46d3-abb5-67a174d38005-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mkp8g\" (UID: \"1af8bc5e-c050-46d3-abb5-67a174d38005\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g"
Sep 04 22:52:31.779163 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.779169    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh4sd\" (UniqueName: \"kubernetes.io/projected/1af8bc5e-c050-46d3-abb5-67a174d38005-kube-api-access-rh4sd\") pod \"cert-manager-webhook-6486ffd997-mkp8g\" (UID: \"1af8bc5e-c050-46d3-abb5-67a174d38005\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g"
Sep 04 22:52:31.880514 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.880426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1af8bc5e-c050-46d3-abb5-67a174d38005-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mkp8g\" (UID: \"1af8bc5e-c050-46d3-abb5-67a174d38005\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g"
Sep 04 22:52:31.880514 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.880476    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh4sd\" (UniqueName: \"kubernetes.io/projected/1af8bc5e-c050-46d3-abb5-67a174d38005-kube-api-access-rh4sd\") pod \"cert-manager-webhook-6486ffd997-mkp8g\" (UID: \"1af8bc5e-c050-46d3-abb5-67a174d38005\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g"
Sep 04 22:52:31.888730 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.888705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1af8bc5e-c050-46d3-abb5-67a174d38005-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mkp8g\" (UID: \"1af8bc5e-c050-46d3-abb5-67a174d38005\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g"
Sep 04 22:52:31.888909 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.888892    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh4sd\" (UniqueName: \"kubernetes.io/projected/1af8bc5e-c050-46d3-abb5-67a174d38005-kube-api-access-rh4sd\") pod \"cert-manager-webhook-6486ffd997-mkp8g\" (UID: \"1af8bc5e-c050-46d3-abb5-67a174d38005\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g"
Sep 04 22:52:31.922495 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:31.922457    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g"
Sep 04 22:52:32.190020 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:32.189994    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mkp8g"]
Sep 04 22:52:32.191879 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:52:32.191854    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1af8bc5e_c050_46d3_abb5_67a174d38005.slice/crio-952d0084d63d0773d3f3d28ac4b5743d15cddff3e0dfe27dbcf8abe9ebcb2698 WatchSource:0}: Error finding container 952d0084d63d0773d3f3d28ac4b5743d15cddff3e0dfe27dbcf8abe9ebcb2698: Status 404 returned error can't find the container with id 952d0084d63d0773d3f3d28ac4b5743d15cddff3e0dfe27dbcf8abe9ebcb2698
Sep 04 22:52:32.483738 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:32.483658    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:32.604325 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:32.604293    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:33.018772 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:33.018730    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g" event={"ID":"1af8bc5e-c050-46d3-abb5-67a174d38005","Type":"ContainerStarted","Data":"da7f3a5c7f4c9935a0a9235b8ac80536e023e57ed46ae78f46183a1c98ed8ada"}
Sep 04 22:52:33.018772 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:33.018776    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g" event={"ID":"1af8bc5e-c050-46d3-abb5-67a174d38005","Type":"ContainerStarted","Data":"952d0084d63d0773d3f3d28ac4b5743d15cddff3e0dfe27dbcf8abe9ebcb2698"}
Sep 04 22:52:33.019000 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:33.018900    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g"
Sep 04 22:52:33.036704 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:33.036659    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g" podStartSLOduration=2.03663392 podStartE2EDuration="2.03663392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:52:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:52:33.034552767 +0000 UTC m=+441.144673354" watchObservedRunningTime="2026-09-04 22:52:33.03663392 +0000 UTC m=+441.146754505"
Sep 04 22:52:34.227965 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:34.227937    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:34.333973 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:34.333935    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:34.428521 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:34.428491    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:36.533497 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.533407    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h"]
Sep 04 22:52:36.536656 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.536619    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h"
Sep 04 22:52:36.539033 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.539014    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-f8g5j\""
Sep 04 22:52:36.539135 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.539015    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 04 22:52:36.539929 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.539912    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 04 22:52:36.544897 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.544869    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h"]
Sep 04 22:52:36.618971 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.618936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwl4l\" (UniqueName: \"kubernetes.io/projected/bb7edda7-d710-43d2-a0b6-d9011c1bfb22-kube-api-access-xwl4l\") pod \"openshift-lws-operator-fd8ccff4c-grp6h\" (UID: \"bb7edda7-d710-43d2-a0b6-d9011c1bfb22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h"
Sep 04 22:52:36.619127 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.619020    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb7edda7-d710-43d2-a0b6-d9011c1bfb22-tmp\") pod \"openshift-lws-operator-fd8ccff4c-grp6h\" (UID: \"bb7edda7-d710-43d2-a0b6-d9011c1bfb22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h"
Sep 04 22:52:36.720307 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.720266    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb7edda7-d710-43d2-a0b6-d9011c1bfb22-tmp\") pod \"openshift-lws-operator-fd8ccff4c-grp6h\" (UID: \"bb7edda7-d710-43d2-a0b6-d9011c1bfb22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h"
Sep 04 22:52:36.720466 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.720326    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwl4l\" (UniqueName: \"kubernetes.io/projected/bb7edda7-d710-43d2-a0b6-d9011c1bfb22-kube-api-access-xwl4l\") pod \"openshift-lws-operator-fd8ccff4c-grp6h\" (UID: \"bb7edda7-d710-43d2-a0b6-d9011c1bfb22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h"
Sep 04 22:52:36.720780 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.720754    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb7edda7-d710-43d2-a0b6-d9011c1bfb22-tmp\") pod \"openshift-lws-operator-fd8ccff4c-grp6h\" (UID: \"bb7edda7-d710-43d2-a0b6-d9011c1bfb22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h"
Sep 04 22:52:36.729159 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.729135    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwl4l\" (UniqueName: \"kubernetes.io/projected/bb7edda7-d710-43d2-a0b6-d9011c1bfb22-kube-api-access-xwl4l\") pod \"openshift-lws-operator-fd8ccff4c-grp6h\" (UID: \"bb7edda7-d710-43d2-a0b6-d9011c1bfb22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h"
Sep 04 22:52:36.847054 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:36.847026    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h"
Sep 04 22:52:37.115135 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:37.115046    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h"]
Sep 04 22:52:37.118145 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:52:37.118116    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb7edda7_d710_43d2_a0b6_d9011c1bfb22.slice/crio-3509e4e0e48b7d9fb80c96a2c30fa9dbc0eaff649fe2f04f2bfab11a61360f76 WatchSource:0}: Error finding container 3509e4e0e48b7d9fb80c96a2c30fa9dbc0eaff649fe2f04f2bfab11a61360f76: Status 404 returned error can't find the container with id 3509e4e0e48b7d9fb80c96a2c30fa9dbc0eaff649fe2f04f2bfab11a61360f76
Sep 04 22:52:38.040974 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:38.040936    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h" event={"ID":"bb7edda7-d710-43d2-a0b6-d9011c1bfb22","Type":"ContainerStarted","Data":"3509e4e0e48b7d9fb80c96a2c30fa9dbc0eaff649fe2f04f2bfab11a61360f76"}
Sep 04 22:52:39.025186 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:39.025154    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mkp8g"
Sep 04 22:52:46.079219 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:46.079184    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h" event={"ID":"bb7edda7-d710-43d2-a0b6-d9011c1bfb22","Type":"ContainerStarted","Data":"bcc8f761467d0ffeda87b5801339c3e8a4f878952d7e7fc84ad7b91b965228f6"}
Sep 04 22:52:46.094733 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:46.094681    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-grp6h" podStartSLOduration=1.608278716 podStartE2EDuration="10.0946238s" totalImagesPullingTime="8.486345084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:52:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:52:46.094099229 +0000 UTC m=+454.204219830" watchObservedRunningTime="2026-09-04 22:52:46.0946238 +0000 UTC m=+454.204744387"
Sep 04 22:52:46.560974 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:46.560886    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:46.841353 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:46.841321    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:46.945212 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:46.945177    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:48.604853 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:48.604822    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:52:59.308691 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.308654    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"]
Sep 04 22:52:59.314687 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.314665    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:52:59.316922 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.316902    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 04 22:52:59.317037 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.316989    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 04 22:52:59.318045 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.318003    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nz8jb\""
Sep 04 22:52:59.320057 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.320029    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"]
Sep 04 22:52:59.405235 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.405204    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5\" (UID: \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:52:59.405409 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.405247    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52wg5\" (UniqueName: \"kubernetes.io/projected/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-kube-api-access-52wg5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5\" (UID: \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:52:59.405409 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.405329    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5\" (UID: \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:52:59.506487 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.506448    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5\" (UID: \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:52:59.506777 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.506753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5\" (UID: \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:52:59.506946 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.506929    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52wg5\" (UniqueName: \"kubernetes.io/projected/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-kube-api-access-52wg5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5\" (UID: \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:52:59.507255 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.507081    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5\" (UID: \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:52:59.510659 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.507540    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5\" (UID: \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:52:59.515858 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.515837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52wg5\" (UniqueName: \"kubernetes.io/projected/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-kube-api-access-52wg5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5\" (UID: \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:52:59.624969 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.624937    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:52:59.893462 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:52:59.893438    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"]
Sep 04 22:52:59.895521 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:52:59.895492    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a50eaa1_1cd5_4624_87d5_1c25c55414a3.slice/crio-af7ac62f7c026a5f50058339ab3bfeb76eabe6d964d2655df96b042693ac1600 WatchSource:0}: Error finding container af7ac62f7c026a5f50058339ab3bfeb76eabe6d964d2655df96b042693ac1600: Status 404 returned error can't find the container with id af7ac62f7c026a5f50058339ab3bfeb76eabe6d964d2655df96b042693ac1600
Sep 04 22:53:00.141057 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:00.141016    2578 generic.go:356] "Generic (PLEG): container finished" podID="6a50eaa1-1cd5-4624-87d5-1c25c55414a3" containerID="b475138f91a4980b7a9e7cae1a59c2a0aa1ba018ed426390d90a447384d9bbc2" exitCode=0
Sep 04 22:53:00.141234 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:00.141067    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5" event={"ID":"6a50eaa1-1cd5-4624-87d5-1c25c55414a3","Type":"ContainerDied","Data":"b475138f91a4980b7a9e7cae1a59c2a0aa1ba018ed426390d90a447384d9bbc2"}
Sep 04 22:53:00.141234 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:00.141112    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5" event={"ID":"6a50eaa1-1cd5-4624-87d5-1c25c55414a3","Type":"ContainerStarted","Data":"af7ac62f7c026a5f50058339ab3bfeb76eabe6d964d2655df96b042693ac1600"}
Sep 04 22:53:01.147089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:01.147054    2578 generic.go:356] "Generic (PLEG): container finished" podID="6a50eaa1-1cd5-4624-87d5-1c25c55414a3" containerID="fe098434b327a2fb15b8d15cc9594eae0b41bfed817ccccf567311fd6405e65d" exitCode=0
Sep 04 22:53:01.147476 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:01.147135    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5" event={"ID":"6a50eaa1-1cd5-4624-87d5-1c25c55414a3","Type":"ContainerDied","Data":"fe098434b327a2fb15b8d15cc9594eae0b41bfed817ccccf567311fd6405e65d"}
Sep 04 22:53:02.153214 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:02.153177    2578 generic.go:356] "Generic (PLEG): container finished" podID="6a50eaa1-1cd5-4624-87d5-1c25c55414a3" containerID="1687022c1c5303e7e3b1a5fdee6a557c13a01381eda2b6a1776e3f4bb9fef118" exitCode=0
Sep 04 22:53:02.153583 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:02.153254    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5" event={"ID":"6a50eaa1-1cd5-4624-87d5-1c25c55414a3","Type":"ContainerDied","Data":"1687022c1c5303e7e3b1a5fdee6a557c13a01381eda2b6a1776e3f4bb9fef118"}
Sep 04 22:53:03.322090 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:03.322063    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:53:03.437443 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:03.437366    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52wg5\" (UniqueName: \"kubernetes.io/projected/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-kube-api-access-52wg5\") pod \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\" (UID: \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\") "
Sep 04 22:53:03.437568 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:03.437446    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-util\") pod \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\" (UID: \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\") "
Sep 04 22:53:03.437568 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:03.437473    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-bundle\") pod \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\" (UID: \"6a50eaa1-1cd5-4624-87d5-1c25c55414a3\") "
Sep 04 22:53:03.438336 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:03.438309    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-bundle" (OuterVolumeSpecName: "bundle") pod "6a50eaa1-1cd5-4624-87d5-1c25c55414a3" (UID: "6a50eaa1-1cd5-4624-87d5-1c25c55414a3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:53:03.439503 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:03.439481    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-kube-api-access-52wg5" (OuterVolumeSpecName: "kube-api-access-52wg5") pod "6a50eaa1-1cd5-4624-87d5-1c25c55414a3" (UID: "6a50eaa1-1cd5-4624-87d5-1c25c55414a3"). InnerVolumeSpecName "kube-api-access-52wg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:53:03.443593 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:03.443567    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-util" (OuterVolumeSpecName: "util") pod "6a50eaa1-1cd5-4624-87d5-1c25c55414a3" (UID: "6a50eaa1-1cd5-4624-87d5-1c25c55414a3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:53:03.538023 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:03.537991    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-util\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:53:03.538023 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:03.538017    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-bundle\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:53:03.538023 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:03.538026    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52wg5\" (UniqueName: \"kubernetes.io/projected/6a50eaa1-1cd5-4624-87d5-1c25c55414a3-kube-api-access-52wg5\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:53:04.163185 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:04.163157    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5" event={"ID":"6a50eaa1-1cd5-4624-87d5-1c25c55414a3","Type":"ContainerDied","Data":"af7ac62f7c026a5f50058339ab3bfeb76eabe6d964d2655df96b042693ac1600"}
Sep 04 22:53:04.163314 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:04.163203    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af7ac62f7c026a5f50058339ab3bfeb76eabe6d964d2655df96b042693ac1600"
Sep 04 22:53:04.163314 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:04.163170    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmtg5"
Sep 04 22:53:08.953894 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.953855    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"]
Sep 04 22:53:08.954267 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.954144    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a50eaa1-1cd5-4624-87d5-1c25c55414a3" containerName="util"
Sep 04 22:53:08.954267 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.954155    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a50eaa1-1cd5-4624-87d5-1c25c55414a3" containerName="util"
Sep 04 22:53:08.954267 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.954166    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a50eaa1-1cd5-4624-87d5-1c25c55414a3" containerName="pull"
Sep 04 22:53:08.954267 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.954171    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a50eaa1-1cd5-4624-87d5-1c25c55414a3" containerName="pull"
Sep 04 22:53:08.954267 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.954180    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a50eaa1-1cd5-4624-87d5-1c25c55414a3" containerName="extract"
Sep 04 22:53:08.954267 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.954186    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a50eaa1-1cd5-4624-87d5-1c25c55414a3" containerName="extract"
Sep 04 22:53:08.954267 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.954241    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a50eaa1-1cd5-4624-87d5-1c25c55414a3" containerName="extract"
Sep 04 22:53:08.957222 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.957203    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:08.959985 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.959962    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 04 22:53:08.960891 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.960875    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 04 22:53:08.961004 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.960913    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nz8jb\""
Sep 04 22:53:08.967403 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:08.967377    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"]
Sep 04 22:53:09.083321 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.083289    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a28656f0-8fad-46b0-b340-f36078a54914-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62\" (UID: \"a28656f0-8fad-46b0-b340-f36078a54914\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:09.083467 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.083332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a28656f0-8fad-46b0-b340-f36078a54914-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62\" (UID: \"a28656f0-8fad-46b0-b340-f36078a54914\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:09.083467 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.083357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbjmc\" (UniqueName: \"kubernetes.io/projected/a28656f0-8fad-46b0-b340-f36078a54914-kube-api-access-cbjmc\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62\" (UID: \"a28656f0-8fad-46b0-b340-f36078a54914\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:09.184160 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.184117    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a28656f0-8fad-46b0-b340-f36078a54914-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62\" (UID: \"a28656f0-8fad-46b0-b340-f36078a54914\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:09.184324 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.184175    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a28656f0-8fad-46b0-b340-f36078a54914-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62\" (UID: \"a28656f0-8fad-46b0-b340-f36078a54914\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:09.184324 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.184215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbjmc\" (UniqueName: \"kubernetes.io/projected/a28656f0-8fad-46b0-b340-f36078a54914-kube-api-access-cbjmc\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62\" (UID: \"a28656f0-8fad-46b0-b340-f36078a54914\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:09.184521 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.184502    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a28656f0-8fad-46b0-b340-f36078a54914-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62\" (UID: \"a28656f0-8fad-46b0-b340-f36078a54914\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:09.184575 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.184556    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a28656f0-8fad-46b0-b340-f36078a54914-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62\" (UID: \"a28656f0-8fad-46b0-b340-f36078a54914\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:09.193143 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.193113    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbjmc\" (UniqueName: \"kubernetes.io/projected/a28656f0-8fad-46b0-b340-f36078a54914-kube-api-access-cbjmc\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62\" (UID: \"a28656f0-8fad-46b0-b340-f36078a54914\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:09.266820 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.266731    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:09.537373 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.537347    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"]
Sep 04 22:53:09.540085 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:53:09.540053    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda28656f0_8fad_46b0_b340_f36078a54914.slice/crio-c98127d04fe7402bb73be4ca9945a10795e2b7b6857f0c1c7e57075217549552 WatchSource:0}: Error finding container c98127d04fe7402bb73be4ca9945a10795e2b7b6857f0c1c7e57075217549552: Status 404 returned error can't find the container with id c98127d04fe7402bb73be4ca9945a10795e2b7b6857f0c1c7e57075217549552
Sep 04 22:53:09.715799 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.715766    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:09.816046 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:09.815965    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:10.189339 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.189302    2578 generic.go:356] "Generic (PLEG): container finished" podID="a28656f0-8fad-46b0-b340-f36078a54914" containerID="3e8dc4965ebb0c14e5581df26a7cdc2213031043e9c6171a5b122996af71c771" exitCode=0
Sep 04 22:53:10.189339 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.189339    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62" event={"ID":"a28656f0-8fad-46b0-b340-f36078a54914","Type":"ContainerDied","Data":"3e8dc4965ebb0c14e5581df26a7cdc2213031043e9c6171a5b122996af71c771"}
Sep 04 22:53:10.189892 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.189373    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62" event={"ID":"a28656f0-8fad-46b0-b340-f36078a54914","Type":"ContainerStarted","Data":"c98127d04fe7402bb73be4ca9945a10795e2b7b6857f0c1c7e57075217549552"}
Sep 04 22:53:10.307782 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.307751    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:10.482422 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.482331    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:10.583015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.582984    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:10.615692 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.615658    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"]
Sep 04 22:53:10.619320 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.619297    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.625014 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.624970    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 04 22:53:10.625014 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.625008    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 04 22:53:10.625206 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.624983    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 04 22:53:10.625206 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.625090    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d78dr\""
Sep 04 22:53:10.625335 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.625320    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 04 22:53:10.641818 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.641794    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"]
Sep 04 22:53:10.696728 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.696684    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/924c2f81-6665-473d-9e70-39cc1baee267-manifests\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.696901 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.696777    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/924c2f81-6665-473d-9e70-39cc1baee267-apiservice-cert\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.696901 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.696808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/924c2f81-6665-473d-9e70-39cc1baee267-webhook-cert\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.696901 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.696874    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/924c2f81-6665-473d-9e70-39cc1baee267-tmp\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.696901 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.696891    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm86g\" (UniqueName: \"kubernetes.io/projected/924c2f81-6665-473d-9e70-39cc1baee267-kube-api-access-hm86g\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.769578 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.769534    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:10.797729 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.797701    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/924c2f81-6665-473d-9e70-39cc1baee267-apiservice-cert\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.797839 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.797732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/924c2f81-6665-473d-9e70-39cc1baee267-webhook-cert\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.797839 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.797760    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/924c2f81-6665-473d-9e70-39cc1baee267-tmp\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.797839 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.797776    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm86g\" (UniqueName: \"kubernetes.io/projected/924c2f81-6665-473d-9e70-39cc1baee267-kube-api-access-hm86g\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.797839 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.797822    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/924c2f81-6665-473d-9e70-39cc1baee267-manifests\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.798160 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.798141    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/924c2f81-6665-473d-9e70-39cc1baee267-manifests\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.799976 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.799944    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/924c2f81-6665-473d-9e70-39cc1baee267-tmp\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.800139 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.800110    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/924c2f81-6665-473d-9e70-39cc1baee267-webhook-cert\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.800197 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.800159    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/924c2f81-6665-473d-9e70-39cc1baee267-apiservice-cert\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.805673 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.805618    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm86g\" (UniqueName: \"kubernetes.io/projected/924c2f81-6665-473d-9e70-39cc1baee267-kube-api-access-hm86g\") pod \"opendatahub-operator-controller-manager-759c5d6957-4qdnw\" (UID: \"924c2f81-6665-473d-9e70-39cc1baee267\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.870694 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.870660    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:10.929321 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.929287    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:10.971332 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:10.971294    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:11.071501 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:11.071412    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:11.183712 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:11.183683    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:11.195352 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:11.195322    2578 generic.go:356] "Generic (PLEG): container finished" podID="a28656f0-8fad-46b0-b340-f36078a54914" containerID="c47b65967245cb7c7afe57c7dbd79f509cc57766b5f17708e78a007f477c29ba" exitCode=0
Sep 04 22:53:11.195722 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:11.195364    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62" event={"ID":"a28656f0-8fad-46b0-b340-f36078a54914","Type":"ContainerDied","Data":"c47b65967245cb7c7afe57c7dbd79f509cc57766b5f17708e78a007f477c29ba"}
Sep 04 22:53:11.213838 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:11.213814    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"]
Sep 04 22:53:11.216206 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:53:11.216176    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod924c2f81_6665_473d_9e70_39cc1baee267.slice/crio-999406291a03183ab11f467a3cec24887215b7131a453fc0ec375dfbddd6d37e WatchSource:0}: Error finding container 999406291a03183ab11f467a3cec24887215b7131a453fc0ec375dfbddd6d37e: Status 404 returned error can't find the container with id 999406291a03183ab11f467a3cec24887215b7131a453fc0ec375dfbddd6d37e
Sep 04 22:53:11.278626 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:11.278596    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:11.378127 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:11.378077    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:12.203184 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:12.203138    2578 generic.go:356] "Generic (PLEG): container finished" podID="a28656f0-8fad-46b0-b340-f36078a54914" containerID="8b2ca33fefb7b4aa5e10bcbabb9ad71dd0af9c72cbafb0d4b67ca94306ec4c41" exitCode=0
Sep 04 22:53:12.203623 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:12.203220    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62" event={"ID":"a28656f0-8fad-46b0-b340-f36078a54914","Type":"ContainerDied","Data":"8b2ca33fefb7b4aa5e10bcbabb9ad71dd0af9c72cbafb0d4b67ca94306ec4c41"}
Sep 04 22:53:12.204787 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:12.204758    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw" event={"ID":"924c2f81-6665-473d-9e70-39cc1baee267","Type":"ContainerStarted","Data":"999406291a03183ab11f467a3cec24887215b7131a453fc0ec375dfbddd6d37e"}
Sep 04 22:53:12.897121 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:12.897086    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:13.029800 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:13.029766    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:14.176612 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.176581    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:14.219607 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.219570    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62" event={"ID":"a28656f0-8fad-46b0-b340-f36078a54914","Type":"ContainerDied","Data":"c98127d04fe7402bb73be4ca9945a10795e2b7b6857f0c1c7e57075217549552"}
Sep 04 22:53:14.219737 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.219685    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c98127d04fe7402bb73be4ca9945a10795e2b7b6857f0c1c7e57075217549552"
Sep 04 22:53:14.219737 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.219761    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0r5p62"
Sep 04 22:53:14.228388 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.227260    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbjmc\" (UniqueName: \"kubernetes.io/projected/a28656f0-8fad-46b0-b340-f36078a54914-kube-api-access-cbjmc\") pod \"a28656f0-8fad-46b0-b340-f36078a54914\" (UID: \"a28656f0-8fad-46b0-b340-f36078a54914\") "
Sep 04 22:53:14.228388 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.227354    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a28656f0-8fad-46b0-b340-f36078a54914-bundle\") pod \"a28656f0-8fad-46b0-b340-f36078a54914\" (UID: \"a28656f0-8fad-46b0-b340-f36078a54914\") "
Sep 04 22:53:14.228388 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.227390    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a28656f0-8fad-46b0-b340-f36078a54914-util\") pod \"a28656f0-8fad-46b0-b340-f36078a54914\" (UID: \"a28656f0-8fad-46b0-b340-f36078a54914\") "
Sep 04 22:53:14.229877 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.229835    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28656f0-8fad-46b0-b340-f36078a54914-bundle" (OuterVolumeSpecName: "bundle") pod "a28656f0-8fad-46b0-b340-f36078a54914" (UID: "a28656f0-8fad-46b0-b340-f36078a54914"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:53:14.233846 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.231556    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a28656f0-8fad-46b0-b340-f36078a54914-kube-api-access-cbjmc" (OuterVolumeSpecName: "kube-api-access-cbjmc") pod "a28656f0-8fad-46b0-b340-f36078a54914" (UID: "a28656f0-8fad-46b0-b340-f36078a54914"). InnerVolumeSpecName "kube-api-access-cbjmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:53:14.235080 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.235052    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28656f0-8fad-46b0-b340-f36078a54914-util" (OuterVolumeSpecName: "util") pod "a28656f0-8fad-46b0-b340-f36078a54914" (UID: "a28656f0-8fad-46b0-b340-f36078a54914"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:53:14.328530 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.328502    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a28656f0-8fad-46b0-b340-f36078a54914-bundle\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:53:14.328530 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.328526    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a28656f0-8fad-46b0-b340-f36078a54914-util\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:53:14.328530 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:14.328535    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbjmc\" (UniqueName: \"kubernetes.io/projected/a28656f0-8fad-46b0-b340-f36078a54914-kube-api-access-cbjmc\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:53:15.229050 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:15.229008    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw" event={"ID":"924c2f81-6665-473d-9e70-39cc1baee267","Type":"ContainerStarted","Data":"c3718b3688ab8822894511c8c4ab77b8986c99313305233e579b6112b56edf61"}
Sep 04 22:53:16.235402 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:16.235363    2578 generic.go:356] "Generic (PLEG): container finished" podID="924c2f81-6665-473d-9e70-39cc1baee267" containerID="c3718b3688ab8822894511c8c4ab77b8986c99313305233e579b6112b56edf61" exitCode=0
Sep 04 22:53:16.235834 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:16.235410    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw" event={"ID":"924c2f81-6665-473d-9e70-39cc1baee267","Type":"ContainerDied","Data":"c3718b3688ab8822894511c8c4ab77b8986c99313305233e579b6112b56edf61"}
Sep 04 22:53:17.242506 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:17.242476    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw" event={"ID":"924c2f81-6665-473d-9e70-39cc1baee267","Type":"ContainerStarted","Data":"719a8f4a9cfed5dfba1c463325b2d7ff7747ccfeacb965c43d3fc6fcfdf28206"}
Sep 04 22:53:17.242889 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:17.242592    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:17.266657 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:17.266600    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw" podStartSLOduration=2.89966367 podStartE2EDuration="7.266585823s" totalImagesPullingTime="3.319289884s" totalInitContainerRuntime="1.047632269s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:53:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:53:17.264462112 +0000 UTC m=+485.374582698" watchObservedRunningTime="2026-09-04 22:53:17.266585823 +0000 UTC m=+485.376706409"
Sep 04 22:53:28.249737 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:28.249704    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-4qdnw"
Sep 04 22:53:29.912892 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:29.912853    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:30.009057 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.009025    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:30.195465 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.195380    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:30.220544 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.220500    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"]
Sep 04 22:53:30.220840 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.220817    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a28656f0-8fad-46b0-b340-f36078a54914" containerName="extract"
Sep 04 22:53:30.220934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.220843    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a28656f0-8fad-46b0-b340-f36078a54914" containerName="extract"
Sep 04 22:53:30.220934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.220875    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a28656f0-8fad-46b0-b340-f36078a54914" containerName="util"
Sep 04 22:53:30.220934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.220884    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a28656f0-8fad-46b0-b340-f36078a54914" containerName="util"
Sep 04 22:53:30.220934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.220902    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a28656f0-8fad-46b0-b340-f36078a54914" containerName="pull"
Sep 04 22:53:30.220934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.220910    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a28656f0-8fad-46b0-b340-f36078a54914" containerName="pull"
Sep 04 22:53:30.221164 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.220984    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a28656f0-8fad-46b0-b340-f36078a54914" containerName="extract"
Sep 04 22:53:30.228888 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.228856    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"
Sep 04 22:53:30.231711 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.231687    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 04 22:53:30.231833 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.231689    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 04 22:53:30.231833 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.231804    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-kfxnl\""
Sep 04 22:53:30.240953 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.240928    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"]
Sep 04 22:53:30.367162 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.367124    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ppm8\" (UniqueName: \"kubernetes.io/projected/0f532f7e-424e-425b-9cdb-031bf519aa94-kube-api-access-4ppm8\") pod \"kube-auth-proxy-c4db4689d-7q5ms\" (UID: \"0f532f7e-424e-425b-9cdb-031bf519aa94\") " pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"
Sep 04 22:53:30.367334 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.367205    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f532f7e-424e-425b-9cdb-031bf519aa94-tls-certs\") pod \"kube-auth-proxy-c4db4689d-7q5ms\" (UID: \"0f532f7e-424e-425b-9cdb-031bf519aa94\") " pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"
Sep 04 22:53:30.367334 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.367233    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f532f7e-424e-425b-9cdb-031bf519aa94-tmp\") pod \"kube-auth-proxy-c4db4689d-7q5ms\" (UID: \"0f532f7e-424e-425b-9cdb-031bf519aa94\") " pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"
Sep 04 22:53:30.468575 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.468487    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f532f7e-424e-425b-9cdb-031bf519aa94-tmp\") pod \"kube-auth-proxy-c4db4689d-7q5ms\" (UID: \"0f532f7e-424e-425b-9cdb-031bf519aa94\") " pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"
Sep 04 22:53:30.468575 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.468535    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ppm8\" (UniqueName: \"kubernetes.io/projected/0f532f7e-424e-425b-9cdb-031bf519aa94-kube-api-access-4ppm8\") pod \"kube-auth-proxy-c4db4689d-7q5ms\" (UID: \"0f532f7e-424e-425b-9cdb-031bf519aa94\") " pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"
Sep 04 22:53:30.468788 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.468589    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f532f7e-424e-425b-9cdb-031bf519aa94-tls-certs\") pod \"kube-auth-proxy-c4db4689d-7q5ms\" (UID: \"0f532f7e-424e-425b-9cdb-031bf519aa94\") " pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"
Sep 04 22:53:30.468788 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:53:30.468696    2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 04 22:53:30.468788 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:53:30.468754    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f532f7e-424e-425b-9cdb-031bf519aa94-tls-certs podName:0f532f7e-424e-425b-9cdb-031bf519aa94 nodeName:}" failed. No retries permitted until 2026-09-04 22:53:30.968738152 +0000 UTC m=+499.078858715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0f532f7e-424e-425b-9cdb-031bf519aa94-tls-certs") pod "kube-auth-proxy-c4db4689d-7q5ms" (UID: "0f532f7e-424e-425b-9cdb-031bf519aa94") : secret "kube-auth-proxy-tls" not found
Sep 04 22:53:30.470851 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.470814    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f532f7e-424e-425b-9cdb-031bf519aa94-tmp\") pod \"kube-auth-proxy-c4db4689d-7q5ms\" (UID: \"0f532f7e-424e-425b-9cdb-031bf519aa94\") " pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"
Sep 04 22:53:30.483458 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.483431    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ppm8\" (UniqueName: \"kubernetes.io/projected/0f532f7e-424e-425b-9cdb-031bf519aa94-kube-api-access-4ppm8\") pod \"kube-auth-proxy-c4db4689d-7q5ms\" (UID: \"0f532f7e-424e-425b-9cdb-031bf519aa94\") " pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"
Sep 04 22:53:30.665960 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.665924    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:30.972822 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.972783    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f532f7e-424e-425b-9cdb-031bf519aa94-tls-certs\") pod \"kube-auth-proxy-c4db4689d-7q5ms\" (UID: \"0f532f7e-424e-425b-9cdb-031bf519aa94\") " pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"
Sep 04 22:53:30.975173 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:30.975152    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f532f7e-424e-425b-9cdb-031bf519aa94-tls-certs\") pod \"kube-auth-proxy-c4db4689d-7q5ms\" (UID: \"0f532f7e-424e-425b-9cdb-031bf519aa94\") " pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"
Sep 04 22:53:31.139910 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:31.139870    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"
Sep 04 22:53:31.415293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:31.415270    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms"]
Sep 04 22:53:31.417204 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:53:31.417177    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f532f7e_424e_425b_9cdb_031bf519aa94.slice/crio-607e4c9fcbdc565b116b81f732899e9c9734001a0a14da633184bdc9b9ca3d90 WatchSource:0}: Error finding container 607e4c9fcbdc565b116b81f732899e9c9734001a0a14da633184bdc9b9ca3d90: Status 404 returned error can't find the container with id 607e4c9fcbdc565b116b81f732899e9c9734001a0a14da633184bdc9b9ca3d90
Sep 04 22:53:31.473582 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:31.473552    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:31.540843 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:31.540812    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:32.312127 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:32.312049    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms" event={"ID":"0f532f7e-424e-425b-9cdb-031bf519aa94","Type":"ContainerStarted","Data":"607e4c9fcbdc565b116b81f732899e9c9734001a0a14da633184bdc9b9ca3d90"}
Sep 04 22:53:32.473710 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:32.473670    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:35.328426 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:35.328390    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms" event={"ID":"0f532f7e-424e-425b-9cdb-031bf519aa94","Type":"ContainerStarted","Data":"a9fb32b9277c07bba6d2ce6cfc0485fa3d6f996032efcb0d1961b7b67c02fe2c"}
Sep 04 22:53:35.345684 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:35.345593    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-c4db4689d-7q5ms" podStartSLOduration=2.20813425 podStartE2EDuration="5.345577888s" totalImagesPullingTime="3.137443638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:53:35.343993805 +0000 UTC m=+503.454118462" watchObservedRunningTime="2026-09-04 22:53:35.345577888 +0000 UTC m=+503.455698473"
Sep 04 22:53:36.489560 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:36.489521    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:36.594543 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:36.594513    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:37.065024 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:37.064993    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:37.269382 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:37.269350    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:37.369120 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:37.369090    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:40.870363 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:40.870322    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-4j6mw"]
Sep 04 22:53:40.873586 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:40.873568    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw"
Sep 04 22:53:40.876403 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:40.876378    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-pbjvn\""
Sep 04 22:53:40.876682 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:40.876462    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 04 22:53:40.883456 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:40.883432    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-4j6mw"]
Sep 04 22:53:40.891666 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:40.891627    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:40.965878 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:40.965841    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lrr8\" (UniqueName: \"kubernetes.io/projected/69a2aca3-4bef-4b6e-9a7b-af150ef0a71f-kube-api-access-5lrr8\") pod \"odh-model-controller-7998c67dc4-4j6mw\" (UID: \"69a2aca3-4bef-4b6e-9a7b-af150ef0a71f\") " pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw"
Sep 04 22:53:40.966026 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:40.965900    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69a2aca3-4bef-4b6e-9a7b-af150ef0a71f-cert\") pod \"odh-model-controller-7998c67dc4-4j6mw\" (UID: \"69a2aca3-4bef-4b6e-9a7b-af150ef0a71f\") " pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw"
Sep 04 22:53:40.991747 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:40.991714    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:41.066896 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:41.066857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69a2aca3-4bef-4b6e-9a7b-af150ef0a71f-cert\") pod \"odh-model-controller-7998c67dc4-4j6mw\" (UID: \"69a2aca3-4bef-4b6e-9a7b-af150ef0a71f\") " pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw"
Sep 04 22:53:41.067056 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:41.066932    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lrr8\" (UniqueName: \"kubernetes.io/projected/69a2aca3-4bef-4b6e-9a7b-af150ef0a71f-kube-api-access-5lrr8\") pod \"odh-model-controller-7998c67dc4-4j6mw\" (UID: \"69a2aca3-4bef-4b6e-9a7b-af150ef0a71f\") " pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw"
Sep 04 22:53:41.067056 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:53:41.066999    2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 04 22:53:41.067135 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:53:41.067062    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69a2aca3-4bef-4b6e-9a7b-af150ef0a71f-cert podName:69a2aca3-4bef-4b6e-9a7b-af150ef0a71f nodeName:}" failed. No retries permitted until 2026-09-04 22:53:41.567047167 +0000 UTC m=+509.677167731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/69a2aca3-4bef-4b6e-9a7b-af150ef0a71f-cert") pod "odh-model-controller-7998c67dc4-4j6mw" (UID: "69a2aca3-4bef-4b6e-9a7b-af150ef0a71f") : secret "odh-model-controller-webhook-cert" not found
Sep 04 22:53:41.077591 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:41.077562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lrr8\" (UniqueName: \"kubernetes.io/projected/69a2aca3-4bef-4b6e-9a7b-af150ef0a71f-kube-api-access-5lrr8\") pod \"odh-model-controller-7998c67dc4-4j6mw\" (UID: \"69a2aca3-4bef-4b6e-9a7b-af150ef0a71f\") " pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw"
Sep 04 22:53:41.091068 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:41.091039    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:41.269013 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:41.268923    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:41.348308 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:41.348265    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:41.569911 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:41.569826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69a2aca3-4bef-4b6e-9a7b-af150ef0a71f-cert\") pod \"odh-model-controller-7998c67dc4-4j6mw\" (UID: \"69a2aca3-4bef-4b6e-9a7b-af150ef0a71f\") " pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw"
Sep 04 22:53:41.572194 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:41.572163    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69a2aca3-4bef-4b6e-9a7b-af150ef0a71f-cert\") pod \"odh-model-controller-7998c67dc4-4j6mw\" (UID: \"69a2aca3-4bef-4b6e-9a7b-af150ef0a71f\") " pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw"
Sep 04 22:53:41.786558 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:41.786511    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw"
Sep 04 22:53:41.967258 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:41.967220    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:42.060720 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:42.060693    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-4j6mw"]
Sep 04 22:53:42.062629 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:53:42.062601    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69a2aca3_4bef_4b6e_9a7b_af150ef0a71f.slice/crio-7e63436ae2fedbea16be185452661675f6cff1f5204446498da0a4daa3392d30 WatchSource:0}: Error finding container 7e63436ae2fedbea16be185452661675f6cff1f5204446498da0a4daa3392d30: Status 404 returned error can't find the container with id 7e63436ae2fedbea16be185452661675f6cff1f5204446498da0a4daa3392d30
Sep 04 22:53:42.361650 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:42.361615    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw" event={"ID":"69a2aca3-4bef-4b6e-9a7b-af150ef0a71f","Type":"ContainerStarted","Data":"7e63436ae2fedbea16be185452661675f6cff1f5204446498da0a4daa3392d30"}
Sep 04 22:53:43.789911 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:43.789876    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:45.379632 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:45.379594    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw" event={"ID":"69a2aca3-4bef-4b6e-9a7b-af150ef0a71f","Type":"ContainerStarted","Data":"caec467cd41152db9ff72906665edab2f1fb30baf2ad6422ad7b9cfdeef56560"}
Sep 04 22:53:45.380008 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:45.379784    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw"
Sep 04 22:53:45.395334 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:45.395288    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw" podStartSLOduration=2.765719923 podStartE2EDuration="5.395274981s" totalImagesPullingTime="2.629555058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:53:45.39434297 +0000 UTC m=+513.504463554" watchObservedRunningTime="2026-09-04 22:53:45.395274981 +0000 UTC m=+513.505395566"
Sep 04 22:53:48.189899 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.189859    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"]
Sep 04 22:53:48.196690 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.196662    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.199215 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.199187    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 04 22:53:48.199215 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.199203    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 04 22:53:48.199420 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.199204    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 04 22:53:48.199420 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.199299    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-n5s97\""
Sep 04 22:53:48.199577 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.199562    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 04 22:53:48.206604 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.206583    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"]
Sep 04 22:53:48.221063 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.221033    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/60baba46-17cd-4e43-a0c7-013389ecf4be-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.221305 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.221276    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/60baba46-17cd-4e43-a0c7-013389ecf4be-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.221433 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.221333    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60baba46-17cd-4e43-a0c7-013389ecf4be-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.221433 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.221410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/60baba46-17cd-4e43-a0c7-013389ecf4be-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.221540 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.221445    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx6wc\" (UniqueName: \"kubernetes.io/projected/60baba46-17cd-4e43-a0c7-013389ecf4be-kube-api-access-mx6wc\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.221540 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.221499    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60baba46-17cd-4e43-a0c7-013389ecf4be-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.221658 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.221534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/60baba46-17cd-4e43-a0c7-013389ecf4be-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.322174 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.322138    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/60baba46-17cd-4e43-a0c7-013389ecf4be-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.322174 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.322176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60baba46-17cd-4e43-a0c7-013389ecf4be-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.322334 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.322214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/60baba46-17cd-4e43-a0c7-013389ecf4be-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.322334 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.322236    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx6wc\" (UniqueName: \"kubernetes.io/projected/60baba46-17cd-4e43-a0c7-013389ecf4be-kube-api-access-mx6wc\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.322334 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.322263    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60baba46-17cd-4e43-a0c7-013389ecf4be-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.322334 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.322282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/60baba46-17cd-4e43-a0c7-013389ecf4be-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.322334 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.322312    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/60baba46-17cd-4e43-a0c7-013389ecf4be-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.323200 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.323146    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/60baba46-17cd-4e43-a0c7-013389ecf4be-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.324578 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.324544    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/60baba46-17cd-4e43-a0c7-013389ecf4be-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.324779 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.324754    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/60baba46-17cd-4e43-a0c7-013389ecf4be-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.324823 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.324766    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/60baba46-17cd-4e43-a0c7-013389ecf4be-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.324859 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.324840    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60baba46-17cd-4e43-a0c7-013389ecf4be-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.331965 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.331944    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60baba46-17cd-4e43-a0c7-013389ecf4be-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.331965 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.331960    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx6wc\" (UniqueName: \"kubernetes.io/projected/60baba46-17cd-4e43-a0c7-013389ecf4be-kube-api-access-mx6wc\") pod \"istiod-openshift-gateway-5677ccb49b-ljj6v\" (UID: \"60baba46-17cd-4e43-a0c7-013389ecf4be\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.507361 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.507278    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:48.783989 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:48.783959    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"]
Sep 04 22:53:48.786199 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:53:48.786172    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60baba46_17cd_4e43_a0c7_013389ecf4be.slice/crio-68d4943c8cc009a6ba75ab23944fd945f65026dc3da61b6c97f594e003f7b8b4 WatchSource:0}: Error finding container 68d4943c8cc009a6ba75ab23944fd945f65026dc3da61b6c97f594e003f7b8b4: Status 404 returned error can't find the container with id 68d4943c8cc009a6ba75ab23944fd945f65026dc3da61b6c97f594e003f7b8b4
Sep 04 22:53:49.402220 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:49.402175    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v" event={"ID":"60baba46-17cd-4e43-a0c7-013389ecf4be","Type":"ContainerStarted","Data":"68d4943c8cc009a6ba75ab23944fd945f65026dc3da61b6c97f594e003f7b8b4"}
Sep 04 22:53:50.776657 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:50.776592    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:50.886323 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:50.886286    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:50.995811 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:50.995779    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:51.067253 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:51.067211    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 22:53:51.067349 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:51.067278    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 22:53:51.414315 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:51.414276    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v" event={"ID":"60baba46-17cd-4e43-a0c7-013389ecf4be","Type":"ContainerStarted","Data":"caad443f0a11830c734cec496e590276215748274948522e11c1991443a20af2"}
Sep 04 22:53:51.414481 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:51.414327    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:51.455226 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:51.455177    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v" podStartSLOduration=1.177031456 podStartE2EDuration="3.455162514s" totalImagesPullingTime="2.278131058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:53:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:53:51.452145664 +0000 UTC m=+519.562266228" watchObservedRunningTime="2026-09-04 22:53:51.455162514 +0000 UTC m=+519.565283100"
Sep 04 22:53:51.741255 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:51.741148    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:52.156862 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:52.156830    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:52.249284 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:52.249250    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:52.317078 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:52.317044    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:52.421355 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:52.421276    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ljj6v"
Sep 04 22:53:52.514221 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:52.514187    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:52.597824 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:52.597787    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:56.149627 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:56.149585    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:56.250693 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:56.250660    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:53:56.386477 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:56.386441    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-4j6mw"
Sep 04 22:53:56.963156 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:53:56.963113    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:01.345675 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:01.345621    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:01.406924 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:01.406892    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:42.081490 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:42.081453    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:43.354452 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:43.354416    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:45.423566 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.423528    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"]
Sep 04 22:54:45.428557 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.428531    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:45.430996 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.430974    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nz8jb\""
Sep 04 22:54:45.431109 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.430984    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 04 22:54:45.431815 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.431799    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 04 22:54:45.434498 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.434479    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"]
Sep 04 22:54:45.524121 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.524083    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"]
Sep 04 22:54:45.525197 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.525173    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px27t\" (UniqueName: \"kubernetes.io/projected/5673ca01-e6ad-4361-b845-fc5277da23c3-kube-api-access-px27t\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq\" (UID: \"5673ca01-e6ad-4361-b845-fc5277da23c3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:45.525269 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.525237    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5673ca01-e6ad-4361-b845-fc5277da23c3-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq\" (UID: \"5673ca01-e6ad-4361-b845-fc5277da23c3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:45.525269 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.525265    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5673ca01-e6ad-4361-b845-fc5277da23c3-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq\" (UID: \"5673ca01-e6ad-4361-b845-fc5277da23c3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:45.527406 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.527389    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:45.541217 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.537752    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"]
Sep 04 22:54:45.622835 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.622793    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"]
Sep 04 22:54:45.626130 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.626098    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tnlj\" (UniqueName: \"kubernetes.io/projected/46e8462e-a801-40dd-b91e-24437c66c374-kube-api-access-7tnlj\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8\" (UID: \"46e8462e-a801-40dd-b91e-24437c66c374\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:45.626271 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.626140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px27t\" (UniqueName: \"kubernetes.io/projected/5673ca01-e6ad-4361-b845-fc5277da23c3-kube-api-access-px27t\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq\" (UID: \"5673ca01-e6ad-4361-b845-fc5277da23c3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:45.626271 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.626186    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46e8462e-a801-40dd-b91e-24437c66c374-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8\" (UID: \"46e8462e-a801-40dd-b91e-24437c66c374\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:45.626271 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.626231    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5673ca01-e6ad-4361-b845-fc5277da23c3-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq\" (UID: \"5673ca01-e6ad-4361-b845-fc5277da23c3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:45.626271 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.626261    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46e8462e-a801-40dd-b91e-24437c66c374-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8\" (UID: \"46e8462e-a801-40dd-b91e-24437c66c374\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:45.626498 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.626278    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5673ca01-e6ad-4361-b845-fc5277da23c3-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq\" (UID: \"5673ca01-e6ad-4361-b845-fc5277da23c3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:45.626498 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.626308    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:45.626602 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.626585    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5673ca01-e6ad-4361-b845-fc5277da23c3-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq\" (UID: \"5673ca01-e6ad-4361-b845-fc5277da23c3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:45.626602 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.626583    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5673ca01-e6ad-4361-b845-fc5277da23c3-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq\" (UID: \"5673ca01-e6ad-4361-b845-fc5277da23c3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:45.634851 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.634815    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"]
Sep 04 22:54:45.637581 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.637557    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px27t\" (UniqueName: \"kubernetes.io/projected/5673ca01-e6ad-4361-b845-fc5277da23c3-kube-api-access-px27t\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq\" (UID: \"5673ca01-e6ad-4361-b845-fc5277da23c3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:45.726942 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.726859    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"]
Sep 04 22:54:45.727217 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.727191    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46e8462e-a801-40dd-b91e-24437c66c374-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8\" (UID: \"46e8462e-a801-40dd-b91e-24437c66c374\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:45.727266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.727236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctlvb\" (UniqueName: \"kubernetes.io/projected/3b9ee480-0d89-4536-b313-5eac38ae084e-kube-api-access-ctlvb\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl\" (UID: \"3b9ee480-0d89-4536-b313-5eac38ae084e\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:45.727313 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.727298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tnlj\" (UniqueName: \"kubernetes.io/projected/46e8462e-a801-40dd-b91e-24437c66c374-kube-api-access-7tnlj\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8\" (UID: \"46e8462e-a801-40dd-b91e-24437c66c374\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:45.727364 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.727336    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b9ee480-0d89-4536-b313-5eac38ae084e-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl\" (UID: \"3b9ee480-0d89-4536-b313-5eac38ae084e\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:45.727420 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.727388    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b9ee480-0d89-4536-b313-5eac38ae084e-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl\" (UID: \"3b9ee480-0d89-4536-b313-5eac38ae084e\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:45.727420 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.727415    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46e8462e-a801-40dd-b91e-24437c66c374-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8\" (UID: \"46e8462e-a801-40dd-b91e-24437c66c374\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:45.727588 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.727565    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46e8462e-a801-40dd-b91e-24437c66c374-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8\" (UID: \"46e8462e-a801-40dd-b91e-24437c66c374\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:45.727714 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.727696    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46e8462e-a801-40dd-b91e-24437c66c374-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8\" (UID: \"46e8462e-a801-40dd-b91e-24437c66c374\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:45.730537 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.730522    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:45.735505 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.735478    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tnlj\" (UniqueName: \"kubernetes.io/projected/46e8462e-a801-40dd-b91e-24437c66c374-kube-api-access-7tnlj\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8\" (UID: \"46e8462e-a801-40dd-b91e-24437c66c374\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:45.739015 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.738994    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:45.739770 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.739749    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"]
Sep 04 22:54:45.827846 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.827817    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b9ee480-0d89-4536-b313-5eac38ae084e-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl\" (UID: \"3b9ee480-0d89-4536-b313-5eac38ae084e\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:45.828017 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.827859    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6cwm\" (UniqueName: \"kubernetes.io/projected/c022e906-a7f4-47b8-81a4-275c7f3294cc-kube-api-access-r6cwm\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s\" (UID: \"c022e906-a7f4-47b8-81a4-275c7f3294cc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:45.828017 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.827954    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c022e906-a7f4-47b8-81a4-275c7f3294cc-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s\" (UID: \"c022e906-a7f4-47b8-81a4-275c7f3294cc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:45.828155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.828026    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b9ee480-0d89-4536-b313-5eac38ae084e-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl\" (UID: \"3b9ee480-0d89-4536-b313-5eac38ae084e\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:45.828155 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.828121    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctlvb\" (UniqueName: \"kubernetes.io/projected/3b9ee480-0d89-4536-b313-5eac38ae084e-kube-api-access-ctlvb\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl\" (UID: \"3b9ee480-0d89-4536-b313-5eac38ae084e\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:45.828271 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.828196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c022e906-a7f4-47b8-81a4-275c7f3294cc-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s\" (UID: \"c022e906-a7f4-47b8-81a4-275c7f3294cc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:45.828271 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.828255    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b9ee480-0d89-4536-b313-5eac38ae084e-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl\" (UID: \"3b9ee480-0d89-4536-b313-5eac38ae084e\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:45.828353 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.828305    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b9ee480-0d89-4536-b313-5eac38ae084e-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl\" (UID: \"3b9ee480-0d89-4536-b313-5eac38ae084e\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:45.836097 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.836059    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctlvb\" (UniqueName: \"kubernetes.io/projected/3b9ee480-0d89-4536-b313-5eac38ae084e-kube-api-access-ctlvb\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl\" (UID: \"3b9ee480-0d89-4536-b313-5eac38ae084e\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:45.843383 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.843358    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:45.928741 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.928701    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c022e906-a7f4-47b8-81a4-275c7f3294cc-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s\" (UID: \"c022e906-a7f4-47b8-81a4-275c7f3294cc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:45.928906 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.928797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6cwm\" (UniqueName: \"kubernetes.io/projected/c022e906-a7f4-47b8-81a4-275c7f3294cc-kube-api-access-r6cwm\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s\" (UID: \"c022e906-a7f4-47b8-81a4-275c7f3294cc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:45.928906 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.928841    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c022e906-a7f4-47b8-81a4-275c7f3294cc-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s\" (UID: \"c022e906-a7f4-47b8-81a4-275c7f3294cc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:45.929147 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.929123    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c022e906-a7f4-47b8-81a4-275c7f3294cc-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s\" (UID: \"c022e906-a7f4-47b8-81a4-275c7f3294cc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:45.929289 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.929258    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c022e906-a7f4-47b8-81a4-275c7f3294cc-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s\" (UID: \"c022e906-a7f4-47b8-81a4-275c7f3294cc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:45.936456 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.936424    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:45.938988 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:45.938960    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6cwm\" (UniqueName: \"kubernetes.io/projected/c022e906-a7f4-47b8-81a4-275c7f3294cc-kube-api-access-r6cwm\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s\" (UID: \"c022e906-a7f4-47b8-81a4-275c7f3294cc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:46.019457 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.019431    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"]
Sep 04 22:54:46.021841 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:54:46.021816    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5673ca01_e6ad_4361_b845_fc5277da23c3.slice/crio-e4fee22da65d2d81c993b79300abecbe5a48d3960fe1d0add11a0cb965e671d4 WatchSource:0}: Error finding container e4fee22da65d2d81c993b79300abecbe5a48d3960fe1d0add11a0cb965e671d4: Status 404 returned error can't find the container with id e4fee22da65d2d81c993b79300abecbe5a48d3960fe1d0add11a0cb965e671d4
Sep 04 22:54:46.048715 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.048691    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:46.117161 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.117132    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"]
Sep 04 22:54:46.119014 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:54:46.118989    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e8462e_a801_40dd_b91e_24437c66c374.slice/crio-5c77b91ccb30b38533a72516efd007d8354cf89483b95081d7f217d34b492e7c WatchSource:0}: Error finding container 5c77b91ccb30b38533a72516efd007d8354cf89483b95081d7f217d34b492e7c: Status 404 returned error can't find the container with id 5c77b91ccb30b38533a72516efd007d8354cf89483b95081d7f217d34b492e7c
Sep 04 22:54:46.212702 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.212674    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"]
Sep 04 22:54:46.213941 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:54:46.213913    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b9ee480_0d89_4536_b313_5eac38ae084e.slice/crio-45dd4a112261cb5fc07353f30cacae8012a7498c7b0366a6fcde9110cde4c50d WatchSource:0}: Error finding container 45dd4a112261cb5fc07353f30cacae8012a7498c7b0366a6fcde9110cde4c50d: Status 404 returned error can't find the container with id 45dd4a112261cb5fc07353f30cacae8012a7498c7b0366a6fcde9110cde4c50d
Sep 04 22:54:46.326707 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.326679    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"]
Sep 04 22:54:46.365482 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:54:46.365449    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc022e906_a7f4_47b8_81a4_275c7f3294cc.slice/crio-4073313179d579f7345fdc29b5b6cb2dda794298e6c09f734d38ca29ea0a5414 WatchSource:0}: Error finding container 4073313179d579f7345fdc29b5b6cb2dda794298e6c09f734d38ca29ea0a5414: Status 404 returned error can't find the container with id 4073313179d579f7345fdc29b5b6cb2dda794298e6c09f734d38ca29ea0a5414
Sep 04 22:54:46.672134 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.672103    2578 generic.go:356] "Generic (PLEG): container finished" podID="3b9ee480-0d89-4536-b313-5eac38ae084e" containerID="6e6aec80ca0538cf9eae28713c0be5b36d970069c23eabf8a35282785fe2ba90" exitCode=0
Sep 04 22:54:46.672601 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.672187    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl" event={"ID":"3b9ee480-0d89-4536-b313-5eac38ae084e","Type":"ContainerDied","Data":"6e6aec80ca0538cf9eae28713c0be5b36d970069c23eabf8a35282785fe2ba90"}
Sep 04 22:54:46.672601 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.672245    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl" event={"ID":"3b9ee480-0d89-4536-b313-5eac38ae084e","Type":"ContainerStarted","Data":"45dd4a112261cb5fc07353f30cacae8012a7498c7b0366a6fcde9110cde4c50d"}
Sep 04 22:54:46.673884 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.673860    2578 generic.go:356] "Generic (PLEG): container finished" podID="5673ca01-e6ad-4361-b845-fc5277da23c3" containerID="4994e10ea686e2ce3c9b4c051748d17a47259dde29de3bb4ea83102b596d08c8" exitCode=0
Sep 04 22:54:46.673958 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.673939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq" event={"ID":"5673ca01-e6ad-4361-b845-fc5277da23c3","Type":"ContainerDied","Data":"4994e10ea686e2ce3c9b4c051748d17a47259dde29de3bb4ea83102b596d08c8"}
Sep 04 22:54:46.674029 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.673984    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq" event={"ID":"5673ca01-e6ad-4361-b845-fc5277da23c3","Type":"ContainerStarted","Data":"e4fee22da65d2d81c993b79300abecbe5a48d3960fe1d0add11a0cb965e671d4"}
Sep 04 22:54:46.675567 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.675543    2578 generic.go:356] "Generic (PLEG): container finished" podID="c022e906-a7f4-47b8-81a4-275c7f3294cc" containerID="0cdac085365758a232683ae7eb755e83e6a2efeb43868ddbc0d1a611329953f9" exitCode=0
Sep 04 22:54:46.675775 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.675674    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s" event={"ID":"c022e906-a7f4-47b8-81a4-275c7f3294cc","Type":"ContainerDied","Data":"0cdac085365758a232683ae7eb755e83e6a2efeb43868ddbc0d1a611329953f9"}
Sep 04 22:54:46.675775 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.675720    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s" event={"ID":"c022e906-a7f4-47b8-81a4-275c7f3294cc","Type":"ContainerStarted","Data":"4073313179d579f7345fdc29b5b6cb2dda794298e6c09f734d38ca29ea0a5414"}
Sep 04 22:54:46.677411 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.677392    2578 generic.go:356] "Generic (PLEG): container finished" podID="46e8462e-a801-40dd-b91e-24437c66c374" containerID="ebbcd856f38f78d6b10d67967a9c7636486615b678d7877be51d99ec79db5ef9" exitCode=0
Sep 04 22:54:46.677498 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.677453    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8" event={"ID":"46e8462e-a801-40dd-b91e-24437c66c374","Type":"ContainerDied","Data":"ebbcd856f38f78d6b10d67967a9c7636486615b678d7877be51d99ec79db5ef9"}
Sep 04 22:54:46.677498 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:46.677490    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8" event={"ID":"46e8462e-a801-40dd-b91e-24437c66c374","Type":"ContainerStarted","Data":"5c77b91ccb30b38533a72516efd007d8354cf89483b95081d7f217d34b492e7c"}
Sep 04 22:54:47.377447 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:47.377408    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:47.476360 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:47.476321    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:47.577673 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:47.577629    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:47.687917 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:47.687884    2578 generic.go:356] "Generic (PLEG): container finished" podID="46e8462e-a801-40dd-b91e-24437c66c374" containerID="c7693ab530ef0734bb6abd5c64420b3102863f5a698434da880135570d36e91f" exitCode=0
Sep 04 22:54:47.688315 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:47.687968    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8" event={"ID":"46e8462e-a801-40dd-b91e-24437c66c374","Type":"ContainerDied","Data":"c7693ab530ef0734bb6abd5c64420b3102863f5a698434da880135570d36e91f"}
Sep 04 22:54:47.690759 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:47.690733    2578 generic.go:356] "Generic (PLEG): container finished" podID="3b9ee480-0d89-4536-b313-5eac38ae084e" containerID="918e3aab3cdca423cbe26e6595e45b95e950b2b6610214b5a972b91375ce2938" exitCode=0
Sep 04 22:54:47.690880 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:47.690824    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl" event={"ID":"3b9ee480-0d89-4536-b313-5eac38ae084e","Type":"ContainerDied","Data":"918e3aab3cdca423cbe26e6595e45b95e950b2b6610214b5a972b91375ce2938"}
Sep 04 22:54:47.693336 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:47.693284    2578 generic.go:356] "Generic (PLEG): container finished" podID="5673ca01-e6ad-4361-b845-fc5277da23c3" containerID="1079486caeb0b825794cd5ad058be40713c9f0fe43836fd9e4d0d6a60db2fed1" exitCode=0
Sep 04 22:54:47.693401 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:47.693367    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq" event={"ID":"5673ca01-e6ad-4361-b845-fc5277da23c3","Type":"ContainerDied","Data":"1079486caeb0b825794cd5ad058be40713c9f0fe43836fd9e4d0d6a60db2fed1"}
Sep 04 22:54:48.121071 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:48.121039    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:48.700077 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:48.700044    2578 generic.go:356] "Generic (PLEG): container finished" podID="3b9ee480-0d89-4536-b313-5eac38ae084e" containerID="d988f5416638707628aa2fca5078be536ade6fe662eb68fc9c2ac304ed9d2b6a" exitCode=0
Sep 04 22:54:48.700450 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:48.700122    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl" event={"ID":"3b9ee480-0d89-4536-b313-5eac38ae084e","Type":"ContainerDied","Data":"d988f5416638707628aa2fca5078be536ade6fe662eb68fc9c2ac304ed9d2b6a"}
Sep 04 22:54:48.702250 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:48.702223    2578 generic.go:356] "Generic (PLEG): container finished" podID="5673ca01-e6ad-4361-b845-fc5277da23c3" containerID="32895ee2fb924428c8e5b043efbed5e5ed8c014ce1bed5733ea7b74ae1d4c53b" exitCode=0
Sep 04 22:54:48.702355 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:48.702311    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq" event={"ID":"5673ca01-e6ad-4361-b845-fc5277da23c3","Type":"ContainerDied","Data":"32895ee2fb924428c8e5b043efbed5e5ed8c014ce1bed5733ea7b74ae1d4c53b"}
Sep 04 22:54:48.704143 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:48.704120    2578 generic.go:356] "Generic (PLEG): container finished" podID="c022e906-a7f4-47b8-81a4-275c7f3294cc" containerID="5f08f6a9773215f26bda382a8901fcf71a607c0f0d8bbf56a07ebc1307c10172" exitCode=0
Sep 04 22:54:48.704244 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:48.704206    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s" event={"ID":"c022e906-a7f4-47b8-81a4-275c7f3294cc","Type":"ContainerDied","Data":"5f08f6a9773215f26bda382a8901fcf71a607c0f0d8bbf56a07ebc1307c10172"}
Sep 04 22:54:48.706721 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:48.706698    2578 generic.go:356] "Generic (PLEG): container finished" podID="46e8462e-a801-40dd-b91e-24437c66c374" containerID="d5a41f056a83c79c1be6cfd2ca53027b22fd6c3b3e23ed33130375310cc5c5e6" exitCode=0
Sep 04 22:54:48.706828 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:48.706742    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8" event={"ID":"46e8462e-a801-40dd-b91e-24437c66c374","Type":"ContainerDied","Data":"d5a41f056a83c79c1be6cfd2ca53027b22fd6c3b3e23ed33130375310cc5c5e6"}
Sep 04 22:54:49.714414 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.714376    2578 generic.go:356] "Generic (PLEG): container finished" podID="c022e906-a7f4-47b8-81a4-275c7f3294cc" containerID="bfc7dc5e597975269f9f464054337738323619ae33f2c43e1ec2a8beff07326f" exitCode=0
Sep 04 22:54:49.714865 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.714464    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s" event={"ID":"c022e906-a7f4-47b8-81a4-275c7f3294cc","Type":"ContainerDied","Data":"bfc7dc5e597975269f9f464054337738323619ae33f2c43e1ec2a8beff07326f"}
Sep 04 22:54:49.863746 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.863713    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:49.888808 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.888779    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:49.900584 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.900557    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:49.962991 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.962959    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b9ee480-0d89-4536-b313-5eac38ae084e-util\") pod \"3b9ee480-0d89-4536-b313-5eac38ae084e\" (UID: \"3b9ee480-0d89-4536-b313-5eac38ae084e\") "
Sep 04 22:54:49.963161 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.963019    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctlvb\" (UniqueName: \"kubernetes.io/projected/3b9ee480-0d89-4536-b313-5eac38ae084e-kube-api-access-ctlvb\") pod \"3b9ee480-0d89-4536-b313-5eac38ae084e\" (UID: \"3b9ee480-0d89-4536-b313-5eac38ae084e\") "
Sep 04 22:54:49.963161 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.963036    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b9ee480-0d89-4536-b313-5eac38ae084e-bundle\") pod \"3b9ee480-0d89-4536-b313-5eac38ae084e\" (UID: \"3b9ee480-0d89-4536-b313-5eac38ae084e\") "
Sep 04 22:54:49.963161 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.963057    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5673ca01-e6ad-4361-b845-fc5277da23c3-util\") pod \"5673ca01-e6ad-4361-b845-fc5277da23c3\" (UID: \"5673ca01-e6ad-4361-b845-fc5277da23c3\") "
Sep 04 22:54:49.963161 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.963078    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px27t\" (UniqueName: \"kubernetes.io/projected/5673ca01-e6ad-4361-b845-fc5277da23c3-kube-api-access-px27t\") pod \"5673ca01-e6ad-4361-b845-fc5277da23c3\" (UID: \"5673ca01-e6ad-4361-b845-fc5277da23c3\") "
Sep 04 22:54:49.963161 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.963133    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5673ca01-e6ad-4361-b845-fc5277da23c3-bundle\") pod \"5673ca01-e6ad-4361-b845-fc5277da23c3\" (UID: \"5673ca01-e6ad-4361-b845-fc5277da23c3\") "
Sep 04 22:54:49.963161 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.963152    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46e8462e-a801-40dd-b91e-24437c66c374-bundle\") pod \"46e8462e-a801-40dd-b91e-24437c66c374\" (UID: \"46e8462e-a801-40dd-b91e-24437c66c374\") "
Sep 04 22:54:49.963480 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.963169    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46e8462e-a801-40dd-b91e-24437c66c374-util\") pod \"46e8462e-a801-40dd-b91e-24437c66c374\" (UID: \"46e8462e-a801-40dd-b91e-24437c66c374\") "
Sep 04 22:54:49.963480 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.963202    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tnlj\" (UniqueName: \"kubernetes.io/projected/46e8462e-a801-40dd-b91e-24437c66c374-kube-api-access-7tnlj\") pod \"46e8462e-a801-40dd-b91e-24437c66c374\" (UID: \"46e8462e-a801-40dd-b91e-24437c66c374\") "
Sep 04 22:54:49.963772 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.963677    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5673ca01-e6ad-4361-b845-fc5277da23c3-bundle" (OuterVolumeSpecName: "bundle") pod "5673ca01-e6ad-4361-b845-fc5277da23c3" (UID: "5673ca01-e6ad-4361-b845-fc5277da23c3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:54:49.963772 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.963716    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9ee480-0d89-4536-b313-5eac38ae084e-bundle" (OuterVolumeSpecName: "bundle") pod "3b9ee480-0d89-4536-b313-5eac38ae084e" (UID: "3b9ee480-0d89-4536-b313-5eac38ae084e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:54:49.964130 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.964034    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e8462e-a801-40dd-b91e-24437c66c374-bundle" (OuterVolumeSpecName: "bundle") pod "46e8462e-a801-40dd-b91e-24437c66c374" (UID: "46e8462e-a801-40dd-b91e-24437c66c374"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:54:49.965536 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.965478    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46e8462e-a801-40dd-b91e-24437c66c374-kube-api-access-7tnlj" (OuterVolumeSpecName: "kube-api-access-7tnlj") pod "46e8462e-a801-40dd-b91e-24437c66c374" (UID: "46e8462e-a801-40dd-b91e-24437c66c374"). InnerVolumeSpecName "kube-api-access-7tnlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:54:49.965684 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.965632    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b9ee480-0d89-4536-b313-5eac38ae084e-kube-api-access-ctlvb" (OuterVolumeSpecName: "kube-api-access-ctlvb") pod "3b9ee480-0d89-4536-b313-5eac38ae084e" (UID: "3b9ee480-0d89-4536-b313-5eac38ae084e"). InnerVolumeSpecName "kube-api-access-ctlvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:54:49.965778 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.965756    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5673ca01-e6ad-4361-b845-fc5277da23c3-kube-api-access-px27t" (OuterVolumeSpecName: "kube-api-access-px27t") pod "5673ca01-e6ad-4361-b845-fc5277da23c3" (UID: "5673ca01-e6ad-4361-b845-fc5277da23c3"). InnerVolumeSpecName "kube-api-access-px27t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:54:49.969698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.969677    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5673ca01-e6ad-4361-b845-fc5277da23c3-util" (OuterVolumeSpecName: "util") pod "5673ca01-e6ad-4361-b845-fc5277da23c3" (UID: "5673ca01-e6ad-4361-b845-fc5277da23c3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:54:49.971811 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.971787    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e8462e-a801-40dd-b91e-24437c66c374-util" (OuterVolumeSpecName: "util") pod "46e8462e-a801-40dd-b91e-24437c66c374" (UID: "46e8462e-a801-40dd-b91e-24437c66c374"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:54:49.972024 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:49.972008    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9ee480-0d89-4536-b313-5eac38ae084e-util" (OuterVolumeSpecName: "util") pod "3b9ee480-0d89-4536-b313-5eac38ae084e" (UID: "3b9ee480-0d89-4536-b313-5eac38ae084e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:54:50.064125 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.064096    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tnlj\" (UniqueName: \"kubernetes.io/projected/46e8462e-a801-40dd-b91e-24437c66c374-kube-api-access-7tnlj\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:54:50.064125 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.064122    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b9ee480-0d89-4536-b313-5eac38ae084e-util\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:54:50.064125 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.064131    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctlvb\" (UniqueName: \"kubernetes.io/projected/3b9ee480-0d89-4536-b313-5eac38ae084e-kube-api-access-ctlvb\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:54:50.064329 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.064139    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b9ee480-0d89-4536-b313-5eac38ae084e-bundle\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:54:50.064329 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.064153    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5673ca01-e6ad-4361-b845-fc5277da23c3-util\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:54:50.064329 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.064163    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px27t\" (UniqueName: \"kubernetes.io/projected/5673ca01-e6ad-4361-b845-fc5277da23c3-kube-api-access-px27t\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:54:50.064329 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.064171    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5673ca01-e6ad-4361-b845-fc5277da23c3-bundle\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:54:50.064329 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.064180    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46e8462e-a801-40dd-b91e-24437c66c374-bundle\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:54:50.064329 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.064188    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46e8462e-a801-40dd-b91e-24437c66c374-util\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:54:50.721282 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.721196    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8"
Sep 04 22:54:50.721282 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.721218    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0h2fj8" event={"ID":"46e8462e-a801-40dd-b91e-24437c66c374","Type":"ContainerDied","Data":"5c77b91ccb30b38533a72516efd007d8354cf89483b95081d7f217d34b492e7c"}
Sep 04 22:54:50.721282 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.721270    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c77b91ccb30b38533a72516efd007d8354cf89483b95081d7f217d34b492e7c"
Sep 04 22:54:50.723188 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.723168    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl"
Sep 04 22:54:50.723328 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.723163    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183czs7tl" event={"ID":"3b9ee480-0d89-4536-b313-5eac38ae084e","Type":"ContainerDied","Data":"45dd4a112261cb5fc07353f30cacae8012a7498c7b0366a6fcde9110cde4c50d"}
Sep 04 22:54:50.723328 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.723270    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45dd4a112261cb5fc07353f30cacae8012a7498c7b0366a6fcde9110cde4c50d"
Sep 04 22:54:50.725357 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.725338    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq"
Sep 04 22:54:50.725467 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.725375    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2jxcq" event={"ID":"5673ca01-e6ad-4361-b845-fc5277da23c3","Type":"ContainerDied","Data":"e4fee22da65d2d81c993b79300abecbe5a48d3960fe1d0add11a0cb965e671d4"}
Sep 04 22:54:50.725467 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.725415    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4fee22da65d2d81c993b79300abecbe5a48d3960fe1d0add11a0cb965e671d4"
Sep 04 22:54:50.854107 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.854080    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:50.969598 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.969560    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c022e906-a7f4-47b8-81a4-275c7f3294cc-util\") pod \"c022e906-a7f4-47b8-81a4-275c7f3294cc\" (UID: \"c022e906-a7f4-47b8-81a4-275c7f3294cc\") "
Sep 04 22:54:50.969598 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.969602    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6cwm\" (UniqueName: \"kubernetes.io/projected/c022e906-a7f4-47b8-81a4-275c7f3294cc-kube-api-access-r6cwm\") pod \"c022e906-a7f4-47b8-81a4-275c7f3294cc\" (UID: \"c022e906-a7f4-47b8-81a4-275c7f3294cc\") "
Sep 04 22:54:50.969833 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.969671    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c022e906-a7f4-47b8-81a4-275c7f3294cc-bundle\") pod \"c022e906-a7f4-47b8-81a4-275c7f3294cc\" (UID: \"c022e906-a7f4-47b8-81a4-275c7f3294cc\") "
Sep 04 22:54:50.970263 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.970239    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c022e906-a7f4-47b8-81a4-275c7f3294cc-bundle" (OuterVolumeSpecName: "bundle") pod "c022e906-a7f4-47b8-81a4-275c7f3294cc" (UID: "c022e906-a7f4-47b8-81a4-275c7f3294cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:54:50.971733 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.971681    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c022e906-a7f4-47b8-81a4-275c7f3294cc-kube-api-access-r6cwm" (OuterVolumeSpecName: "kube-api-access-r6cwm") pod "c022e906-a7f4-47b8-81a4-275c7f3294cc" (UID: "c022e906-a7f4-47b8-81a4-275c7f3294cc"). InnerVolumeSpecName "kube-api-access-r6cwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:54:50.975070 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:50.975036    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c022e906-a7f4-47b8-81a4-275c7f3294cc-util" (OuterVolumeSpecName: "util") pod "c022e906-a7f4-47b8-81a4-275c7f3294cc" (UID: "c022e906-a7f4-47b8-81a4-275c7f3294cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:54:51.070241 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:51.070206    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c022e906-a7f4-47b8-81a4-275c7f3294cc-util\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:54:51.070241 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:51.070236    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6cwm\" (UniqueName: \"kubernetes.io/projected/c022e906-a7f4-47b8-81a4-275c7f3294cc-kube-api-access-r6cwm\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:54:51.070241 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:51.070249    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c022e906-a7f4-47b8-81a4-275c7f3294cc-bundle\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:54:51.732933 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:51.732896    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s" event={"ID":"c022e906-a7f4-47b8-81a4-275c7f3294cc","Type":"ContainerDied","Data":"4073313179d579f7345fdc29b5b6cb2dda794298e6c09f734d38ca29ea0a5414"}
Sep 04 22:54:51.733397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:51.732948    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4073313179d579f7345fdc29b5b6cb2dda794298e6c09f734d38ca29ea0a5414"
Sep 04 22:54:51.733397 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:51.732946    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537q47s"
Sep 04 22:54:55.108872 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:55.108836    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:57.153522 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153485    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-hhv67"]
Sep 04 22:54:57.153912 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153869    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9ee480-0d89-4536-b313-5eac38ae084e" containerName="extract"
Sep 04 22:54:57.153912 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153883    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9ee480-0d89-4536-b313-5eac38ae084e" containerName="extract"
Sep 04 22:54:57.153912 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153891    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c022e906-a7f4-47b8-81a4-275c7f3294cc" containerName="extract"
Sep 04 22:54:57.153912 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153896    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c022e906-a7f4-47b8-81a4-275c7f3294cc" containerName="extract"
Sep 04 22:54:57.153912 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153905    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5673ca01-e6ad-4361-b845-fc5277da23c3" containerName="util"
Sep 04 22:54:57.153912 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153910    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5673ca01-e6ad-4361-b845-fc5277da23c3" containerName="util"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153919    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5673ca01-e6ad-4361-b845-fc5277da23c3" containerName="extract"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153924    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5673ca01-e6ad-4361-b845-fc5277da23c3" containerName="extract"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153932    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46e8462e-a801-40dd-b91e-24437c66c374" containerName="util"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153937    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="46e8462e-a801-40dd-b91e-24437c66c374" containerName="util"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153946    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46e8462e-a801-40dd-b91e-24437c66c374" containerName="pull"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153951    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="46e8462e-a801-40dd-b91e-24437c66c374" containerName="pull"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153956    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c022e906-a7f4-47b8-81a4-275c7f3294cc" containerName="pull"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153960    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c022e906-a7f4-47b8-81a4-275c7f3294cc" containerName="pull"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153966    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5673ca01-e6ad-4361-b845-fc5277da23c3" containerName="pull"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153972    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5673ca01-e6ad-4361-b845-fc5277da23c3" containerName="pull"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153983    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9ee480-0d89-4536-b313-5eac38ae084e" containerName="util"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153988    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9ee480-0d89-4536-b313-5eac38ae084e" containerName="util"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.153995    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c022e906-a7f4-47b8-81a4-275c7f3294cc" containerName="util"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.154000    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c022e906-a7f4-47b8-81a4-275c7f3294cc" containerName="util"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.154005    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9ee480-0d89-4536-b313-5eac38ae084e" containerName="pull"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.154010    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9ee480-0d89-4536-b313-5eac38ae084e" containerName="pull"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.154016    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46e8462e-a801-40dd-b91e-24437c66c374" containerName="extract"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.154021    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="46e8462e-a801-40dd-b91e-24437c66c374" containerName="extract"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.154069    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b9ee480-0d89-4536-b313-5eac38ae084e" containerName="extract"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.154076    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5673ca01-e6ad-4361-b845-fc5277da23c3" containerName="extract"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.154086    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="46e8462e-a801-40dd-b91e-24437c66c374" containerName="extract"
Sep 04 22:54:57.154104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.154095    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c022e906-a7f4-47b8-81a4-275c7f3294cc" containerName="extract"
Sep 04 22:54:57.158203 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.158183    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-hhv67"
Sep 04 22:54:57.161669 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.161616    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hz97d\""
Sep 04 22:54:57.162118 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.162100    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 04 22:54:57.162545 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.162516    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 04 22:54:57.162608 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.162565    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 04 22:54:57.175386 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.175364    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-hhv67"]
Sep 04 22:54:57.224748 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.224710    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwznd\" (UniqueName: \"kubernetes.io/projected/705222be-4838-479d-b8b8-ac69c00e8147-kube-api-access-hwznd\") pod \"dns-operator-controller-manager-5985564776-hhv67\" (UID: \"705222be-4838-479d-b8b8-ac69c00e8147\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-hhv67"
Sep 04 22:54:57.325427 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.325382    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwznd\" (UniqueName: \"kubernetes.io/projected/705222be-4838-479d-b8b8-ac69c00e8147-kube-api-access-hwznd\") pod \"dns-operator-controller-manager-5985564776-hhv67\" (UID: \"705222be-4838-479d-b8b8-ac69c00e8147\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-hhv67"
Sep 04 22:54:57.343744 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.343710    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwznd\" (UniqueName: \"kubernetes.io/projected/705222be-4838-479d-b8b8-ac69c00e8147-kube-api-access-hwznd\") pod \"dns-operator-controller-manager-5985564776-hhv67\" (UID: \"705222be-4838-479d-b8b8-ac69c00e8147\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-hhv67"
Sep 04 22:54:57.468042 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.467958    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-hhv67"
Sep 04 22:54:57.497428 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.497388    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:57.747915 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.747890    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-hhv67"]
Sep 04 22:54:57.750249 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:54:57.750220    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod705222be_4838_479d_b8b8_ac69c00e8147.slice/crio-ad5085ec325a367baf431958c0c726414475a3fc327f87a9e3b6e7c028182f5c WatchSource:0}: Error finding container ad5085ec325a367baf431958c0c726414475a3fc327f87a9e3b6e7c028182f5c: Status 404 returned error can't find the container with id ad5085ec325a367baf431958c0c726414475a3fc327f87a9e3b6e7c028182f5c
Sep 04 22:54:57.769699 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.769668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-hhv67" event={"ID":"705222be-4838-479d-b8b8-ac69c00e8147","Type":"ContainerStarted","Data":"ad5085ec325a367baf431958c0c726414475a3fc327f87a9e3b6e7c028182f5c"}
Sep 04 22:54:57.908719 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.908688    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:57.932888 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:57.932859    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66fcd75bcd-rds9z"]
Sep 04 22:54:58.072875 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:58.072795    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:58.171760 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:58.171728    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:58.328861 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:58.328830    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:58.428089 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:58.428056    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:59.366732 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:59.366703    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:59.479980 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:59.479939    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:54:59.636723 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:59.636653    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-nf42w"]
Sep 04 22:54:59.645536 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:59.645509    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-nf42w"
Sep 04 22:54:59.649025 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:59.648827    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-fph4t\""
Sep 04 22:54:59.651747 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:59.651717    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-nf42w"]
Sep 04 22:54:59.747896 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:59.747858    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqd4w\" (UniqueName: \"kubernetes.io/projected/e9457cc4-c254-4b46-8b34-367d313ce579-kube-api-access-vqd4w\") pod \"authorino-operator-68d88948d8-nf42w\" (UID: \"e9457cc4-c254-4b46-8b34-367d313ce579\") " pod="kuadrant-system/authorino-operator-68d88948d8-nf42w"
Sep 04 22:54:59.849018 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:59.848979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqd4w\" (UniqueName: \"kubernetes.io/projected/e9457cc4-c254-4b46-8b34-367d313ce579-kube-api-access-vqd4w\") pod \"authorino-operator-68d88948d8-nf42w\" (UID: \"e9457cc4-c254-4b46-8b34-367d313ce579\") " pod="kuadrant-system/authorino-operator-68d88948d8-nf42w"
Sep 04 22:54:59.858286 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:59.858257    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqd4w\" (UniqueName: \"kubernetes.io/projected/e9457cc4-c254-4b46-8b34-367d313ce579-kube-api-access-vqd4w\") pod \"authorino-operator-68d88948d8-nf42w\" (UID: \"e9457cc4-c254-4b46-8b34-367d313ce579\") " pod="kuadrant-system/authorino-operator-68d88948d8-nf42w"
Sep 04 22:54:59.959434 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:54:59.959347    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-nf42w"
Sep 04 22:55:00.501344 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:00.501269    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:00.509698 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:00.509672    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-nf42w"]
Sep 04 22:55:00.511840 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:55:00.511811    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9457cc4_c254_4b46_8b34_367d313ce579.slice/crio-1dcf702839f40c4f3fff0a96bbe050b6a90b97182fd4bdd9668e60ac084871a3 WatchSource:0}: Error finding container 1dcf702839f40c4f3fff0a96bbe050b6a90b97182fd4bdd9668e60ac084871a3: Status 404 returned error can't find the container with id 1dcf702839f40c4f3fff0a96bbe050b6a90b97182fd4bdd9668e60ac084871a3
Sep 04 22:55:00.744457 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:00.744420    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:00.792283 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:00.792196    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-hhv67" event={"ID":"705222be-4838-479d-b8b8-ac69c00e8147","Type":"ContainerStarted","Data":"42f00e7e97669d49d7b74dc24ba6f706204eb6a7dfa649f1135e02ef66af273d"}
Sep 04 22:55:00.792443 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:00.792356    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-hhv67"
Sep 04 22:55:00.793533 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:00.793508    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-nf42w" event={"ID":"e9457cc4-c254-4b46-8b34-367d313ce579","Type":"ContainerStarted","Data":"1dcf702839f40c4f3fff0a96bbe050b6a90b97182fd4bdd9668e60ac084871a3"}
Sep 04 22:55:00.819238 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:00.819189    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-hhv67" podStartSLOduration=1.262310901 podStartE2EDuration="3.81917322s" totalImagesPullingTime="2.556862319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:54:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:55:00.816295304 +0000 UTC m=+588.926415890" watchObservedRunningTime="2026-09-04 22:55:00.81917322 +0000 UTC m=+588.929293806"
Sep 04 22:55:02.390909 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:02.390875    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:02.707857 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:02.707770    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:02.807320 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:02.807283    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-nf42w" event={"ID":"e9457cc4-c254-4b46-8b34-367d313ce579","Type":"ContainerStarted","Data":"193b18a7976f5eb81c60194faefd437b53f49fdad54ef887c012048144b32ebb"}
Sep 04 22:55:02.807478 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:02.807415    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-nf42w"
Sep 04 22:55:02.826222 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:02.826174    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-nf42w" podStartSLOduration=2.410671044 podStartE2EDuration="3.82616057s" totalImagesPullingTime="1.415489526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:54:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:55:02.824679877 +0000 UTC m=+590.934800457" watchObservedRunningTime="2026-09-04 22:55:02.82616057 +0000 UTC m=+590.936281156"
Sep 04 22:55:02.967634 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:02.967562    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:03.101228 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:03.101199    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:03.219277 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:03.219188    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:03.502027 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:03.501938    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:03.607571 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:03.607532    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:06.892687 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:06.892634    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"]
Sep 04 22:55:06.896121 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:06.896101    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:06.899058 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:06.899034    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rs8mg\""
Sep 04 22:55:06.910426 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:06.910402    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"]
Sep 04 22:55:07.014448 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:07.014408    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-qf9wt\" (UID: \"70c0b6d0-dfad-45dc-914b-f78aab8a8bc8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:07.014625 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:07.014502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gnqv\" (UniqueName: \"kubernetes.io/projected/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8-kube-api-access-8gnqv\") pod \"kuadrant-operator-controller-manager-58fc56757c-qf9wt\" (UID: \"70c0b6d0-dfad-45dc-914b-f78aab8a8bc8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:07.115195 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:07.115159    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gnqv\" (UniqueName: \"kubernetes.io/projected/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8-kube-api-access-8gnqv\") pod \"kuadrant-operator-controller-manager-58fc56757c-qf9wt\" (UID: \"70c0b6d0-dfad-45dc-914b-f78aab8a8bc8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:07.115315 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:07.115230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-qf9wt\" (UID: \"70c0b6d0-dfad-45dc-914b-f78aab8a8bc8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:07.115583 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:07.115566    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-qf9wt\" (UID: \"70c0b6d0-dfad-45dc-914b-f78aab8a8bc8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:07.128434 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:07.128407    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gnqv\" (UniqueName: \"kubernetes.io/projected/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8-kube-api-access-8gnqv\") pod \"kuadrant-operator-controller-manager-58fc56757c-qf9wt\" (UID: \"70c0b6d0-dfad-45dc-914b-f78aab8a8bc8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:07.206398 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:07.206307    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:07.485033 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:07.485008    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"]
Sep 04 22:55:07.487348 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:55:07.487319    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c0b6d0_dfad_45dc_914b_f78aab8a8bc8.slice/crio-8af7b7fa007ab03e10cd4670d4df337ebb0165b8ca4a6cffcef7bdda6cd833a4 WatchSource:0}: Error finding container 8af7b7fa007ab03e10cd4670d4df337ebb0165b8ca4a6cffcef7bdda6cd833a4: Status 404 returned error can't find the container with id 8af7b7fa007ab03e10cd4670d4df337ebb0165b8ca4a6cffcef7bdda6cd833a4
Sep 04 22:55:07.836852 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:07.836819    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt" event={"ID":"70c0b6d0-dfad-45dc-914b-f78aab8a8bc8","Type":"ContainerStarted","Data":"8af7b7fa007ab03e10cd4670d4df337ebb0165b8ca4a6cffcef7bdda6cd833a4"}
Sep 04 22:55:08.719298 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:08.719260    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:08.819778 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:08.819745    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:11.802769 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:11.802689    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-hhv67"
Sep 04 22:55:11.862786 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:11.862748    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt" event={"ID":"70c0b6d0-dfad-45dc-914b-f78aab8a8bc8","Type":"ContainerStarted","Data":"da95c6710b1273f2c5360ef5840d203404ca56a08fa81b09bf800a892ed20aad"}
Sep 04 22:55:11.862963 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:11.862845    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:11.881270 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:11.881212    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt" podStartSLOduration=1.8408220339999999 podStartE2EDuration="5.881192456s" totalImagesPullingTime="4.040370422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:55:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:55:11.880414076 +0000 UTC m=+599.990534662" watchObservedRunningTime="2026-09-04 22:55:11.881192456 +0000 UTC m=+599.991313253"
Sep 04 22:55:12.348335 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:12.348304    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 22:55:12.348620 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:12.348602    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 22:55:13.815749 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:13.815718    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-nf42w"
Sep 04 22:55:14.154749 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:14.154714    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:14.353035 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:14.353003    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:14.446183 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:14.446091    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:14.541350 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:14.541313    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:14.807762 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:14.807671    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:14.891758 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:14.891725    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:16.328111 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:16.328069    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:16.428608 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:16.428576    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:18.803675 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:18.803617    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:18.904543 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:18.904509    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:21.085281 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:21.085245    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:21.183934 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:21.183900    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:21.285741 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:21.285708    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:22.870183 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:22.870141    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:22.959359 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:22.959316    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66fcd75bcd-rds9z" podUID="40ccec3b-b162-4302-8eb9-f80ffbac05c2" containerName="console" containerID="cri-o://5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e" gracePeriod=15
Sep 04 22:55:23.217697 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.217674    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:55:23.364212 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.364179    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-trusted-ca-bundle\") pod \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") "
Sep 04 22:55:23.364212 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.364218    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-oauth-config\") pod \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") "
Sep 04 22:55:23.364461 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.364240    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-serving-cert\") pod \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") "
Sep 04 22:55:23.364461 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.364260    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-service-ca\") pod \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") "
Sep 04 22:55:23.364461 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.364309    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpsgn\" (UniqueName: \"kubernetes.io/projected/40ccec3b-b162-4302-8eb9-f80ffbac05c2-kube-api-access-xpsgn\") pod \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") "
Sep 04 22:55:23.364461 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.364327    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-config\") pod \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") "
Sep 04 22:55:23.364461 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.364383    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-oauth-serving-cert\") pod \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\" (UID: \"40ccec3b-b162-4302-8eb9-f80ffbac05c2\") "
Sep 04 22:55:23.364769 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.364697    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "40ccec3b-b162-4302-8eb9-f80ffbac05c2" (UID: "40ccec3b-b162-4302-8eb9-f80ffbac05c2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:55:23.364854 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.364827    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-service-ca" (OuterVolumeSpecName: "service-ca") pod "40ccec3b-b162-4302-8eb9-f80ffbac05c2" (UID: "40ccec3b-b162-4302-8eb9-f80ffbac05c2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:55:23.364900 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.364867    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-config" (OuterVolumeSpecName: "console-config") pod "40ccec3b-b162-4302-8eb9-f80ffbac05c2" (UID: "40ccec3b-b162-4302-8eb9-f80ffbac05c2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:55:23.364953 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.364929    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "40ccec3b-b162-4302-8eb9-f80ffbac05c2" (UID: "40ccec3b-b162-4302-8eb9-f80ffbac05c2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:55:23.366559 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.366531    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "40ccec3b-b162-4302-8eb9-f80ffbac05c2" (UID: "40ccec3b-b162-4302-8eb9-f80ffbac05c2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:55:23.366706 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.366568    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "40ccec3b-b162-4302-8eb9-f80ffbac05c2" (UID: "40ccec3b-b162-4302-8eb9-f80ffbac05c2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:55:23.366706 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.366626    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40ccec3b-b162-4302-8eb9-f80ffbac05c2-kube-api-access-xpsgn" (OuterVolumeSpecName: "kube-api-access-xpsgn") pod "40ccec3b-b162-4302-8eb9-f80ffbac05c2" (UID: "40ccec3b-b162-4302-8eb9-f80ffbac05c2"). InnerVolumeSpecName "kube-api-access-xpsgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:55:23.465782 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.465712    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-oauth-serving-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:23.465782 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.465737    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-trusted-ca-bundle\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:23.465782 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.465748    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-oauth-config\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:23.465782 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.465757    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-serving-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:23.465782 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.465766    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:23.465782 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.465775    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpsgn\" (UniqueName: \"kubernetes.io/projected/40ccec3b-b162-4302-8eb9-f80ffbac05c2-kube-api-access-xpsgn\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:23.465782 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.465784    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40ccec3b-b162-4302-8eb9-f80ffbac05c2-console-config\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:23.934435 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.934406    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"]
Sep 04 22:55:23.934854 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.934759    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40ccec3b-b162-4302-8eb9-f80ffbac05c2" containerName="console"
Sep 04 22:55:23.934854 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.934772    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="40ccec3b-b162-4302-8eb9-f80ffbac05c2" containerName="console"
Sep 04 22:55:23.934854 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.934850    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="40ccec3b-b162-4302-8eb9-f80ffbac05c2" containerName="console"
Sep 04 22:55:23.936112 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.936089    2578 generic.go:356] "Generic (PLEG): container finished" podID="40ccec3b-b162-4302-8eb9-f80ffbac05c2" containerID="5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e" exitCode=0
Sep 04 22:55:23.936236 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.936217    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66fcd75bcd-rds9z"
Sep 04 22:55:23.939178 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.939155    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66fcd75bcd-rds9z" event={"ID":"40ccec3b-b162-4302-8eb9-f80ffbac05c2","Type":"ContainerDied","Data":"5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e"}
Sep 04 22:55:23.939282 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.939186    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66fcd75bcd-rds9z" event={"ID":"40ccec3b-b162-4302-8eb9-f80ffbac05c2","Type":"ContainerDied","Data":"ebcf1bb6771cb2163e95bddcd1fd2681a20ea5d6eb5d806f8114f14584052c32"}
Sep 04 22:55:23.939282 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.939212    2578 scope.go:117] "RemoveContainer" containerID="5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e"
Sep 04 22:55:23.939282 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.939268    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:23.950211 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.950193    2578 scope.go:117] "RemoveContainer" containerID="5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e"
Sep 04 22:55:23.950447 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:23.950430    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e\": container with ID starting with 5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e not found: ID does not exist" containerID="5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e"
Sep 04 22:55:23.950531 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.950452    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e"} err="failed to get container status \"5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e\": rpc error: code = NotFound desc = could not find container \"5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e\": container with ID starting with 5b22840fc6f9a011718f711cd57490c1d22cb427a88d9a6a5dbdef5b3b94205e not found: ID does not exist"
Sep 04 22:55:23.957697 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.957677    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"]
Sep 04 22:55:23.988995 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.988974    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66fcd75bcd-rds9z"]
Sep 04 22:55:23.995305 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:23.995286    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66fcd75bcd-rds9z"]
Sep 04 22:55:24.071140 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.071107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sqwx\" (UniqueName: \"kubernetes.io/projected/e1e9cc64-47f4-43c5-896e-1b311fc4a80d-kube-api-access-5sqwx\") pod \"kuadrant-operator-controller-manager-96db98c47-kvlvl\" (UID: \"e1e9cc64-47f4-43c5-896e-1b311fc4a80d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:24.071301 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.071150    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1e9cc64-47f4-43c5-896e-1b311fc4a80d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-kvlvl\" (UID: \"e1e9cc64-47f4-43c5-896e-1b311fc4a80d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:24.172185 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.172145    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sqwx\" (UniqueName: \"kubernetes.io/projected/e1e9cc64-47f4-43c5-896e-1b311fc4a80d-kube-api-access-5sqwx\") pod \"kuadrant-operator-controller-manager-96db98c47-kvlvl\" (UID: \"e1e9cc64-47f4-43c5-896e-1b311fc4a80d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:24.172333 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.172191    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1e9cc64-47f4-43c5-896e-1b311fc4a80d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-kvlvl\" (UID: \"e1e9cc64-47f4-43c5-896e-1b311fc4a80d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:24.172585 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.172567    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1e9cc64-47f4-43c5-896e-1b311fc4a80d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-kvlvl\" (UID: \"e1e9cc64-47f4-43c5-896e-1b311fc4a80d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:24.181589 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.181559    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sqwx\" (UniqueName: \"kubernetes.io/projected/e1e9cc64-47f4-43c5-896e-1b311fc4a80d-kube-api-access-5sqwx\") pod \"kuadrant-operator-controller-manager-96db98c47-kvlvl\" (UID: \"e1e9cc64-47f4-43c5-896e-1b311fc4a80d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:24.251543 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.251475    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:24.422476 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.422442    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40ccec3b-b162-4302-8eb9-f80ffbac05c2" path="/var/lib/kubelet/pods/40ccec3b-b162-4302-8eb9-f80ffbac05c2/volumes"
Sep 04 22:55:24.529742 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.529690    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"]
Sep 04 22:55:24.531984 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:55:24.531955    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e9cc64_47f4_43c5_896e_1b311fc4a80d.slice/crio-a5a1d9cefad4fa477438e9629d1a2b331daf2f682f14e08fb586810259b17823 WatchSource:0}: Error finding container a5a1d9cefad4fa477438e9629d1a2b331daf2f682f14e08fb586810259b17823: Status 404 returned error can't find the container with id a5a1d9cefad4fa477438e9629d1a2b331daf2f682f14e08fb586810259b17823
Sep 04 22:55:24.943544 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.943495    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl" event={"ID":"e1e9cc64-47f4-43c5-896e-1b311fc4a80d","Type":"ContainerStarted","Data":"f5a4515e3b16678b4be55479da1497532021170bd77ae18b565e0ce7a9699dc6"}
Sep 04 22:55:24.944007 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.943567    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:24.944007 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.943586    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl" event={"ID":"e1e9cc64-47f4-43c5-896e-1b311fc4a80d","Type":"ContainerStarted","Data":"a5a1d9cefad4fa477438e9629d1a2b331daf2f682f14e08fb586810259b17823"}
Sep 04 22:55:24.965327 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:24.965274    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl" podStartSLOduration=1.965257553 podStartE2EDuration="1.965257553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:55:24.963207902 +0000 UTC m=+613.073328483" watchObservedRunningTime="2026-09-04 22:55:24.965257553 +0000 UTC m=+613.075378139"
Sep 04 22:55:25.186723 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.186684    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"]
Sep 04 22:55:25.186926 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.186902    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt" podUID="70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" containerName="manager" containerID="cri-o://da95c6710b1273f2c5360ef5840d203404ca56a08fa81b09bf800a892ed20aad" gracePeriod=2
Sep 04 22:55:25.189393 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:25.189360    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-qf9wt\" is forbidden: User \"system:node:ip-10-0-128-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-32.ec2.internal' and this object" podUID="70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:25.191806 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.191781    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"]
Sep 04 22:55:25.209744 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.209714    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"]
Sep 04 22:55:25.222590 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.222406    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"]
Sep 04 22:55:25.223016 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.222975    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" containerName="manager"
Sep 04 22:55:25.223016 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.223005    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" containerName="manager"
Sep 04 22:55:25.223211 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.223131    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" containerName="manager"
Sep 04 22:55:25.226750 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.226725    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"]
Sep 04 22:55:25.226895 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.226841    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"
Sep 04 22:55:25.229503 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:25.229474    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-qf9wt\" is forbidden: User \"system:node:ip-10-0-128-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-32.ec2.internal' and this object" podUID="70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:25.232447 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.232424    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"]
Sep 04 22:55:25.232914 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.232895    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1e9cc64-47f4-43c5-896e-1b311fc4a80d" containerName="manager"
Sep 04 22:55:25.233004 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.232920    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1e9cc64-47f4-43c5-896e-1b311fc4a80d" containerName="manager"
Sep 04 22:55:25.233061 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.233041    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1e9cc64-47f4-43c5-896e-1b311fc4a80d" containerName="manager"
Sep 04 22:55:25.236756 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.236733    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"
Sep 04 22:55:25.241957 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.241934    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"]
Sep 04 22:55:25.262000 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.261908    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"]
Sep 04 22:55:25.263324 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:25.263286    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-qf9wt\" is forbidden: User \"system:node:ip-10-0-128-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-32.ec2.internal' and this object" podUID="70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:25.280686 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.280660    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ttsp\" (UniqueName: \"kubernetes.io/projected/e000a7ea-b934-4b6a-ad60-ea90d2d20f13-kube-api-access-4ttsp\") pod \"kuadrant-operator-controller-manager-58fc56757c-mtgwq\" (UID: \"e000a7ea-b934-4b6a-ad60-ea90d2d20f13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"
Sep 04 22:55:25.280803 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.280703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qck9\" (UniqueName: \"kubernetes.io/projected/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7-kube-api-access-2qck9\") pod \"kuadrant-operator-controller-manager-96db98c47-w4b6m\" (UID: \"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"
Sep 04 22:55:25.280803 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.280733    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e000a7ea-b934-4b6a-ad60-ea90d2d20f13-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-mtgwq\" (UID: \"e000a7ea-b934-4b6a-ad60-ea90d2d20f13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"
Sep 04 22:55:25.280939 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.280799    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-w4b6m\" (UID: \"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"
Sep 04 22:55:25.381245 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.381214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ttsp\" (UniqueName: \"kubernetes.io/projected/e000a7ea-b934-4b6a-ad60-ea90d2d20f13-kube-api-access-4ttsp\") pod \"kuadrant-operator-controller-manager-58fc56757c-mtgwq\" (UID: \"e000a7ea-b934-4b6a-ad60-ea90d2d20f13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"
Sep 04 22:55:25.381400 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.381258    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qck9\" (UniqueName: \"kubernetes.io/projected/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7-kube-api-access-2qck9\") pod \"kuadrant-operator-controller-manager-96db98c47-w4b6m\" (UID: \"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"
Sep 04 22:55:25.381400 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.381282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e000a7ea-b934-4b6a-ad60-ea90d2d20f13-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-mtgwq\" (UID: \"e000a7ea-b934-4b6a-ad60-ea90d2d20f13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"
Sep 04 22:55:25.381400 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.381319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-w4b6m\" (UID: \"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"
Sep 04 22:55:25.381753 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.381727    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e000a7ea-b934-4b6a-ad60-ea90d2d20f13-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-mtgwq\" (UID: \"e000a7ea-b934-4b6a-ad60-ea90d2d20f13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"
Sep 04 22:55:25.381795 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.381750    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-w4b6m\" (UID: \"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"
Sep 04 22:55:25.389719 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.389687    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qck9\" (UniqueName: \"kubernetes.io/projected/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7-kube-api-access-2qck9\") pod \"kuadrant-operator-controller-manager-96db98c47-w4b6m\" (UID: \"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"
Sep 04 22:55:25.390072 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.390053    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ttsp\" (UniqueName: \"kubernetes.io/projected/e000a7ea-b934-4b6a-ad60-ea90d2d20f13-kube-api-access-4ttsp\") pod \"kuadrant-operator-controller-manager-58fc56757c-mtgwq\" (UID: \"e000a7ea-b934-4b6a-ad60-ea90d2d20f13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"
Sep 04 22:55:25.428193 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.428174    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:25.430559 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:25.430538    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-qf9wt\" is forbidden: User \"system:node:ip-10-0-128-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-32.ec2.internal' and this object" podUID="70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:25.481947 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.481889    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8-extensions-socket-volume\") pod \"70c0b6d0-dfad-45dc-914b-f78aab8a8bc8\" (UID: \"70c0b6d0-dfad-45dc-914b-f78aab8a8bc8\") "
Sep 04 22:55:25.481947 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.481922    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gnqv\" (UniqueName: \"kubernetes.io/projected/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8-kube-api-access-8gnqv\") pod \"70c0b6d0-dfad-45dc-914b-f78aab8a8bc8\" (UID: \"70c0b6d0-dfad-45dc-914b-f78aab8a8bc8\") "
Sep 04 22:55:25.482276 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.482258    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" (UID: "70c0b6d0-dfad-45dc-914b-f78aab8a8bc8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:55:25.483987 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.483964    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8-kube-api-access-8gnqv" (OuterVolumeSpecName: "kube-api-access-8gnqv") pod "70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" (UID: "70c0b6d0-dfad-45dc-914b-f78aab8a8bc8"). InnerVolumeSpecName "kube-api-access-8gnqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:55:25.566195 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.566158    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"
Sep 04 22:55:25.575301 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.575276    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"
Sep 04 22:55:25.582438 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.582416    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8-extensions-socket-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:25.582526 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.582441    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gnqv\" (UniqueName: \"kubernetes.io/projected/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8-kube-api-access-8gnqv\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:25.852010 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.851904    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"]
Sep 04 22:55:25.852261 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:55:25.852237    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode000a7ea_b934_4b6a_ad60_ea90d2d20f13.slice/crio-319a2fd101d1d69ab1f4cece0f15e748b485915bdf2102a5a29a0423ac5a0b8c WatchSource:0}: Error finding container 319a2fd101d1d69ab1f4cece0f15e748b485915bdf2102a5a29a0423ac5a0b8c: Status 404 returned error can't find the container with id 319a2fd101d1d69ab1f4cece0f15e748b485915bdf2102a5a29a0423ac5a0b8c
Sep 04 22:55:25.854169 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:55:25.854142    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30ca9a3_edc1_4cd1_842e_37dcc31d90f7.slice/crio-9e246378059673659b0e7454c846ce71bcf374addcf1600da9fa3062463c8f53 WatchSource:0}: Error finding container 9e246378059673659b0e7454c846ce71bcf374addcf1600da9fa3062463c8f53: Status 404 returned error can't find the container with id 9e246378059673659b0e7454c846ce71bcf374addcf1600da9fa3062463c8f53
Sep 04 22:55:25.854315 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.854293    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"]
Sep 04 22:55:25.952722 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.952682    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq" event={"ID":"e000a7ea-b934-4b6a-ad60-ea90d2d20f13","Type":"ContainerStarted","Data":"d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70"}
Sep 04 22:55:25.953174 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.952749    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq" event={"ID":"e000a7ea-b934-4b6a-ad60-ea90d2d20f13","Type":"ContainerStarted","Data":"319a2fd101d1d69ab1f4cece0f15e748b485915bdf2102a5a29a0423ac5a0b8c"}
Sep 04 22:55:25.953174 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.952780    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"
Sep 04 22:55:25.954168 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.954143    2578 generic.go:356] "Generic (PLEG): container finished" podID="70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" containerID="da95c6710b1273f2c5360ef5840d203404ca56a08fa81b09bf800a892ed20aad" exitCode=0
Sep 04 22:55:25.954270 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.954179    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:25.954270 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.954234    2578 scope.go:117] "RemoveContainer" containerID="da95c6710b1273f2c5360ef5840d203404ca56a08fa81b09bf800a892ed20aad"
Sep 04 22:55:25.954869 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:25.954844    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-qf9wt\" is forbidden: User \"system:node:ip-10-0-128-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-32.ec2.internal' and this object" podUID="70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:25.956189 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.956161    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m" event={"ID":"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7","Type":"ContainerStarted","Data":"8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c"}
Sep 04 22:55:25.956305 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.956216    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"
Sep 04 22:55:25.956305 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.956239    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m" event={"ID":"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7","Type":"ContainerStarted","Data":"9e246378059673659b0e7454c846ce71bcf374addcf1600da9fa3062463c8f53"}
Sep 04 22:55:25.956424 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:25.956372    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl" container="manager"
Sep 04 22:55:25.963401 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.963383    2578 scope.go:117] "RemoveContainer" containerID="da95c6710b1273f2c5360ef5840d203404ca56a08fa81b09bf800a892ed20aad"
Sep 04 22:55:25.963684 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:25.963663    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da95c6710b1273f2c5360ef5840d203404ca56a08fa81b09bf800a892ed20aad\": container with ID starting with da95c6710b1273f2c5360ef5840d203404ca56a08fa81b09bf800a892ed20aad not found: ID does not exist" containerID="da95c6710b1273f2c5360ef5840d203404ca56a08fa81b09bf800a892ed20aad"
Sep 04 22:55:25.963772 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.963689    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da95c6710b1273f2c5360ef5840d203404ca56a08fa81b09bf800a892ed20aad"} err="failed to get container status \"da95c6710b1273f2c5360ef5840d203404ca56a08fa81b09bf800a892ed20aad\": rpc error: code = NotFound desc = could not find container \"da95c6710b1273f2c5360ef5840d203404ca56a08fa81b09bf800a892ed20aad\": container with ID starting with da95c6710b1273f2c5360ef5840d203404ca56a08fa81b09bf800a892ed20aad not found: ID does not exist"
Sep 04 22:55:25.981475 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:25.981436    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq" podStartSLOduration=0.981422381 podStartE2EDuration="981.422381ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:55:25.979918505 +0000 UTC m=+614.090039114" watchObservedRunningTime="2026-09-04 22:55:25.981422381 +0000 UTC m=+614.091542967"
Sep 04 22:55:25.982065 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:25.982037    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-kvlvl\" is forbidden: User \"system:node:ip-10-0-128-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-32.ec2.internal' and this object" podUID="e1e9cc64-47f4-43c5-896e-1b311fc4a80d" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:26.007839 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:26.007797    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m" podStartSLOduration=1.007783401 podStartE2EDuration="1.007783401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:55:26.006128426 +0000 UTC m=+614.116249031" watchObservedRunningTime="2026-09-04 22:55:26.007783401 +0000 UTC m=+614.117903987"
Sep 04 22:55:26.008198 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:26.008173    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-qf9wt\" is forbidden: User \"system:node:ip-10-0-128-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-32.ec2.internal' and this object" podUID="70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qf9wt"
Sep 04 22:55:26.419411 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:26.419377    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c0b6d0-dfad-45dc-914b-f78aab8a8bc8" path="/var/lib/kubelet/pods/70c0b6d0-dfad-45dc-914b-f78aab8a8bc8/volumes"
Sep 04 22:55:26.964478 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:26.964442    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl" podUID="e1e9cc64-47f4-43c5-896e-1b311fc4a80d" containerName="manager" containerID="cri-o://f5a4515e3b16678b4be55479da1497532021170bd77ae18b565e0ce7a9699dc6" gracePeriod=2
Sep 04 22:55:27.214403 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.214376    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:27.296101 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.296024    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1e9cc64-47f4-43c5-896e-1b311fc4a80d-extensions-socket-volume\") pod \"e1e9cc64-47f4-43c5-896e-1b311fc4a80d\" (UID: \"e1e9cc64-47f4-43c5-896e-1b311fc4a80d\") "
Sep 04 22:55:27.296101 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.296077    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sqwx\" (UniqueName: \"kubernetes.io/projected/e1e9cc64-47f4-43c5-896e-1b311fc4a80d-kube-api-access-5sqwx\") pod \"e1e9cc64-47f4-43c5-896e-1b311fc4a80d\" (UID: \"e1e9cc64-47f4-43c5-896e-1b311fc4a80d\") "
Sep 04 22:55:27.296464 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.296440    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1e9cc64-47f4-43c5-896e-1b311fc4a80d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e1e9cc64-47f4-43c5-896e-1b311fc4a80d" (UID: "e1e9cc64-47f4-43c5-896e-1b311fc4a80d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:55:27.298090 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.298072    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1e9cc64-47f4-43c5-896e-1b311fc4a80d-kube-api-access-5sqwx" (OuterVolumeSpecName: "kube-api-access-5sqwx") pod "e1e9cc64-47f4-43c5-896e-1b311fc4a80d" (UID: "e1e9cc64-47f4-43c5-896e-1b311fc4a80d"). InnerVolumeSpecName "kube-api-access-5sqwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:55:27.396625 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.396593    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1e9cc64-47f4-43c5-896e-1b311fc4a80d-extensions-socket-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:27.396625 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.396620    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sqwx\" (UniqueName: \"kubernetes.io/projected/e1e9cc64-47f4-43c5-896e-1b311fc4a80d-kube-api-access-5sqwx\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:27.746194 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.746163    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:27.848513 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.848476    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:27.971567 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.971534    2578 generic.go:356] "Generic (PLEG): container finished" podID="e1e9cc64-47f4-43c5-896e-1b311fc4a80d" containerID="f5a4515e3b16678b4be55479da1497532021170bd77ae18b565e0ce7a9699dc6" exitCode=0
Sep 04 22:55:27.971973 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.971582    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:27.971973 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.971621    2578 scope.go:117] "RemoveContainer" containerID="f5a4515e3b16678b4be55479da1497532021170bd77ae18b565e0ce7a9699dc6"
Sep 04 22:55:27.974007 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:27.973979    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-kvlvl\" is forbidden: User \"system:node:ip-10-0-128-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-32.ec2.internal' and this object" podUID="e1e9cc64-47f4-43c5-896e-1b311fc4a80d" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:27.980625 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.980608    2578 scope.go:117] "RemoveContainer" containerID="f5a4515e3b16678b4be55479da1497532021170bd77ae18b565e0ce7a9699dc6"
Sep 04 22:55:27.980919 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:27.980900    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5a4515e3b16678b4be55479da1497532021170bd77ae18b565e0ce7a9699dc6\": container with ID starting with f5a4515e3b16678b4be55479da1497532021170bd77ae18b565e0ce7a9699dc6 not found: ID does not exist" containerID="f5a4515e3b16678b4be55479da1497532021170bd77ae18b565e0ce7a9699dc6"
Sep 04 22:55:27.980972 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:27.980928    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5a4515e3b16678b4be55479da1497532021170bd77ae18b565e0ce7a9699dc6"} err="failed to get container status \"f5a4515e3b16678b4be55479da1497532021170bd77ae18b565e0ce7a9699dc6\": rpc error: code = NotFound desc = could not find container \"f5a4515e3b16678b4be55479da1497532021170bd77ae18b565e0ce7a9699dc6\": container with ID starting with f5a4515e3b16678b4be55479da1497532021170bd77ae18b565e0ce7a9699dc6 not found: ID does not exist"
Sep 04 22:55:27.981920 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:27.981896    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-kvlvl\" is forbidden: User \"system:node:ip-10-0-128-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-32.ec2.internal' and this object" podUID="e1e9cc64-47f4-43c5-896e-1b311fc4a80d" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-kvlvl"
Sep 04 22:55:28.419160 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:28.419127    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1e9cc64-47f4-43c5-896e-1b311fc4a80d" path="/var/lib/kubelet/pods/e1e9cc64-47f4-43c5-896e-1b311fc4a80d/volumes"
Sep 04 22:55:29.765861 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:29.765828    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:33.123008 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:33.122975    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:33.224464 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:33.224429    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:33.522570 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:33.522486    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:33.586330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:33.586297    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:36.938529 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:36.938452    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:36.966475 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:36.966444    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"
Sep 04 22:55:36.967266 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:36.967243    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"
Sep 04 22:55:37.021309 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.021279    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:37.054990 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.054785    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"]
Sep 04 22:55:37.055501 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.055319    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq" podUID="e000a7ea-b934-4b6a-ad60-ea90d2d20f13" containerName="manager" containerID="cri-o://d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70" gracePeriod=10
Sep 04 22:55:37.304475 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.304449    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"
Sep 04 22:55:37.326751 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.326711    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"]
Sep 04 22:55:37.327122 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.327102    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e000a7ea-b934-4b6a-ad60-ea90d2d20f13" containerName="manager"
Sep 04 22:55:37.327202 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.327126    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e000a7ea-b934-4b6a-ad60-ea90d2d20f13" containerName="manager"
Sep 04 22:55:37.327257 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.327210    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e000a7ea-b934-4b6a-ad60-ea90d2d20f13" containerName="manager"
Sep 04 22:55:37.330121 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.330100    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"
Sep 04 22:55:37.372979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.372942    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"]
Sep 04 22:55:37.380876 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.380845    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ttsp\" (UniqueName: \"kubernetes.io/projected/e000a7ea-b934-4b6a-ad60-ea90d2d20f13-kube-api-access-4ttsp\") pod \"e000a7ea-b934-4b6a-ad60-ea90d2d20f13\" (UID: \"e000a7ea-b934-4b6a-ad60-ea90d2d20f13\") "
Sep 04 22:55:37.381060 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.380891    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e000a7ea-b934-4b6a-ad60-ea90d2d20f13-extensions-socket-volume\") pod \"e000a7ea-b934-4b6a-ad60-ea90d2d20f13\" (UID: \"e000a7ea-b934-4b6a-ad60-ea90d2d20f13\") "
Sep 04 22:55:37.381158 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.381135    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgg2m\" (UniqueName: \"kubernetes.io/projected/65624aef-fcac-43d9-86e6-2c6a3d41c4ee-kube-api-access-hgg2m\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-qtz9h\" (UID: \"65624aef-fcac-43d9-86e6-2c6a3d41c4ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"
Sep 04 22:55:37.381294 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.381271    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/65624aef-fcac-43d9-86e6-2c6a3d41c4ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-qtz9h\" (UID: \"65624aef-fcac-43d9-86e6-2c6a3d41c4ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"
Sep 04 22:55:37.381358 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.381329    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e000a7ea-b934-4b6a-ad60-ea90d2d20f13-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e000a7ea-b934-4b6a-ad60-ea90d2d20f13" (UID: "e000a7ea-b934-4b6a-ad60-ea90d2d20f13"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:55:37.382917 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.382892    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e000a7ea-b934-4b6a-ad60-ea90d2d20f13-kube-api-access-4ttsp" (OuterVolumeSpecName: "kube-api-access-4ttsp") pod "e000a7ea-b934-4b6a-ad60-ea90d2d20f13" (UID: "e000a7ea-b934-4b6a-ad60-ea90d2d20f13"). InnerVolumeSpecName "kube-api-access-4ttsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:55:37.481761 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.481675    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgg2m\" (UniqueName: \"kubernetes.io/projected/65624aef-fcac-43d9-86e6-2c6a3d41c4ee-kube-api-access-hgg2m\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-qtz9h\" (UID: \"65624aef-fcac-43d9-86e6-2c6a3d41c4ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"
Sep 04 22:55:37.481902 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.481874    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/65624aef-fcac-43d9-86e6-2c6a3d41c4ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-qtz9h\" (UID: \"65624aef-fcac-43d9-86e6-2c6a3d41c4ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"
Sep 04 22:55:37.481944 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.481921    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ttsp\" (UniqueName: \"kubernetes.io/projected/e000a7ea-b934-4b6a-ad60-ea90d2d20f13-kube-api-access-4ttsp\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:37.481944 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.481930    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e000a7ea-b934-4b6a-ad60-ea90d2d20f13-extensions-socket-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:37.482235 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.482219    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/65624aef-fcac-43d9-86e6-2c6a3d41c4ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-qtz9h\" (UID: \"65624aef-fcac-43d9-86e6-2c6a3d41c4ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"
Sep 04 22:55:37.494031 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.494006    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgg2m\" (UniqueName: \"kubernetes.io/projected/65624aef-fcac-43d9-86e6-2c6a3d41c4ee-kube-api-access-hgg2m\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-qtz9h\" (UID: \"65624aef-fcac-43d9-86e6-2c6a3d41c4ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"
Sep 04 22:55:37.639716 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.639673    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"
Sep 04 22:55:37.924505 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:37.924472    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"]
Sep 04 22:55:37.925709 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:55:37.925685    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65624aef_fcac_43d9_86e6_2c6a3d41c4ee.slice/crio-9b5e2ee6ff40bca698ad0b0cd1c4ee82e34eac2fb92a18d998b1aa17c4fdb1f7 WatchSource:0}: Error finding container 9b5e2ee6ff40bca698ad0b0cd1c4ee82e34eac2fb92a18d998b1aa17c4fdb1f7: Status 404 returned error can't find the container with id 9b5e2ee6ff40bca698ad0b0cd1c4ee82e34eac2fb92a18d998b1aa17c4fdb1f7
Sep 04 22:55:38.035386 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.035350    2578 generic.go:356] "Generic (PLEG): container finished" podID="e000a7ea-b934-4b6a-ad60-ea90d2d20f13" containerID="d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70" exitCode=0
Sep 04 22:55:38.035813 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.035390    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"
Sep 04 22:55:38.035813 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.035434    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq" event={"ID":"e000a7ea-b934-4b6a-ad60-ea90d2d20f13","Type":"ContainerDied","Data":"d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70"}
Sep 04 22:55:38.035813 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.035494    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq" event={"ID":"e000a7ea-b934-4b6a-ad60-ea90d2d20f13","Type":"ContainerDied","Data":"319a2fd101d1d69ab1f4cece0f15e748b485915bdf2102a5a29a0423ac5a0b8c"}
Sep 04 22:55:38.035813 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.035529    2578 scope.go:117] "RemoveContainer" containerID="d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70"
Sep 04 22:55:38.037777 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.037754    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h" event={"ID":"65624aef-fcac-43d9-86e6-2c6a3d41c4ee","Type":"ContainerStarted","Data":"4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7"}
Sep 04 22:55:38.037881 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.037805    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h" event={"ID":"65624aef-fcac-43d9-86e6-2c6a3d41c4ee","Type":"ContainerStarted","Data":"9b5e2ee6ff40bca698ad0b0cd1c4ee82e34eac2fb92a18d998b1aa17c4fdb1f7"}
Sep 04 22:55:38.037953 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.037919    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"
Sep 04 22:55:38.045716 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.045697    2578 scope.go:117] "RemoveContainer" containerID="d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70"
Sep 04 22:55:38.046013 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:38.045994    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70\": container with ID starting with d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70 not found: ID does not exist" containerID="d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70"
Sep 04 22:55:38.046076 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.046022    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70"} err="failed to get container status \"d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70\": rpc error: code = NotFound desc = could not find container \"d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70\": container with ID starting with d12256302aec0013f590f40b2437a829546c2183389dcfe5b5964f702243ed70 not found: ID does not exist"
Sep 04 22:55:38.061723 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.061668    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h" podStartSLOduration=1.061623487 podStartE2EDuration="1.061623487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:55:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:55:38.060194884 +0000 UTC m=+626.170315471" watchObservedRunningTime="2026-09-04 22:55:38.061623487 +0000 UTC m=+626.171744073"
Sep 04 22:55:38.082818 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.082792    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"]
Sep 04 22:55:38.088115 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.088094    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mtgwq"]
Sep 04 22:55:38.418979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:38.418944    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e000a7ea-b934-4b6a-ad60-ea90d2d20f13" path="/var/lib/kubelet/pods/e000a7ea-b934-4b6a-ad60-ea90d2d20f13/volumes"
Sep 04 22:55:40.693712 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:40.693682    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:40.792587 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:40.792552    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:40.893300 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:40.893264    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:49.047796 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:49.047762    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"
Sep 04 22:55:49.103164 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:49.103118    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"]
Sep 04 22:55:49.103442 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:49.103380    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m" podUID="c30ca9a3-edc1-4cd1-842e-37dcc31d90f7" containerName="manager" containerID="cri-o://8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c" gracePeriod=10
Sep 04 22:55:49.354352 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:49.354324    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"
Sep 04 22:55:49.493243 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:49.493208    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7-extensions-socket-volume\") pod \"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7\" (UID: \"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7\") "
Sep 04 22:55:49.493408 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:49.493260    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qck9\" (UniqueName: \"kubernetes.io/projected/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7-kube-api-access-2qck9\") pod \"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7\" (UID: \"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7\") "
Sep 04 22:55:49.493615 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:49.493579    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c30ca9a3-edc1-4cd1-842e-37dcc31d90f7" (UID: "c30ca9a3-edc1-4cd1-842e-37dcc31d90f7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:55:49.495335 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:49.495316    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7-kube-api-access-2qck9" (OuterVolumeSpecName: "kube-api-access-2qck9") pod "c30ca9a3-edc1-4cd1-842e-37dcc31d90f7" (UID: "c30ca9a3-edc1-4cd1-842e-37dcc31d90f7"). InnerVolumeSpecName "kube-api-access-2qck9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:55:49.594104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:49.594018    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qck9\" (UniqueName: \"kubernetes.io/projected/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7-kube-api-access-2qck9\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:49.594104 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:49.594049    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7-extensions-socket-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:55:50.114984 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:50.114948    2578 generic.go:356] "Generic (PLEG): container finished" podID="c30ca9a3-edc1-4cd1-842e-37dcc31d90f7" containerID="8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c" exitCode=0
Sep 04 22:55:50.115375 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:50.114989    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"
Sep 04 22:55:50.115375 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:50.115032    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m" event={"ID":"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7","Type":"ContainerDied","Data":"8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c"}
Sep 04 22:55:50.115375 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:50.115085    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m" event={"ID":"c30ca9a3-edc1-4cd1-842e-37dcc31d90f7","Type":"ContainerDied","Data":"9e246378059673659b0e7454c846ce71bcf374addcf1600da9fa3062463c8f53"}
Sep 04 22:55:50.115375 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:50.115113    2578 scope.go:117] "RemoveContainer" containerID="8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c"
Sep 04 22:55:50.124304 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:50.124287    2578 scope.go:117] "RemoveContainer" containerID="8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c"
Sep 04 22:55:50.124590 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:55:50.124571    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c\": container with ID starting with 8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c not found: ID does not exist" containerID="8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c"
Sep 04 22:55:50.124687 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:50.124605    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c"} err="failed to get container status \"8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c\": rpc error: code = NotFound desc = could not find container \"8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c\": container with ID starting with 8b7974fe0bd5d885071c967cde2b6dc23004ae4a20a689cba1da7f7fb803f03c not found: ID does not exist"
Sep 04 22:55:50.138855 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:50.138831    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"]
Sep 04 22:55:50.147492 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:50.147473    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-w4b6m"]
Sep 04 22:55:50.418991 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:50.418916    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c30ca9a3-edc1-4cd1-842e-37dcc31d90f7" path="/var/lib/kubelet/pods/c30ca9a3-edc1-4cd1-842e-37dcc31d90f7/volumes"
Sep 04 22:55:53.475988 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:53.475952    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:55:54.206102 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:55:54.206068    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:07.440102 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:07.440073    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:09.498184 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.498147    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:09.502072 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.502045    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"]
Sep 04 22:56:09.502386 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.502374    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c30ca9a3-edc1-4cd1-842e-37dcc31d90f7" containerName="manager"
Sep 04 22:56:09.502428 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.502388    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c30ca9a3-edc1-4cd1-842e-37dcc31d90f7" containerName="manager"
Sep 04 22:56:09.502463 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.502446    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c30ca9a3-edc1-4cd1-842e-37dcc31d90f7" containerName="manager"
Sep 04 22:56:09.504330 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.504315    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"
Sep 04 22:56:09.506832 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.506806    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 04 22:56:09.506980 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.506898    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9x9r4\""
Sep 04 22:56:09.517658 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.517609    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"]
Sep 04 22:56:09.594724 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.594684    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"]
Sep 04 22:56:09.605782 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.605749    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:09.672749 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.672706    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vddqc\" (UniqueName: \"kubernetes.io/projected/4588074c-7684-4403-a2a8-e1aedba4c96c-kube-api-access-vddqc\") pod \"limitador-limitador-859c4d96d6-b8dlv\" (UID: \"4588074c-7684-4403-a2a8-e1aedba4c96c\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"
Sep 04 22:56:09.672925 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.672841    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4588074c-7684-4403-a2a8-e1aedba4c96c-config-file\") pod \"limitador-limitador-859c4d96d6-b8dlv\" (UID: \"4588074c-7684-4403-a2a8-e1aedba4c96c\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"
Sep 04 22:56:09.709321 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.709291    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:09.773757 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.773675    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vddqc\" (UniqueName: \"kubernetes.io/projected/4588074c-7684-4403-a2a8-e1aedba4c96c-kube-api-access-vddqc\") pod \"limitador-limitador-859c4d96d6-b8dlv\" (UID: \"4588074c-7684-4403-a2a8-e1aedba4c96c\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"
Sep 04 22:56:09.773907 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.773808    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4588074c-7684-4403-a2a8-e1aedba4c96c-config-file\") pod \"limitador-limitador-859c4d96d6-b8dlv\" (UID: \"4588074c-7684-4403-a2a8-e1aedba4c96c\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"
Sep 04 22:56:09.774415 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.774393    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4588074c-7684-4403-a2a8-e1aedba4c96c-config-file\") pod \"limitador-limitador-859c4d96d6-b8dlv\" (UID: \"4588074c-7684-4403-a2a8-e1aedba4c96c\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"
Sep 04 22:56:09.781677 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.781636    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vddqc\" (UniqueName: \"kubernetes.io/projected/4588074c-7684-4403-a2a8-e1aedba4c96c-kube-api-access-vddqc\") pod \"limitador-limitador-859c4d96d6-b8dlv\" (UID: \"4588074c-7684-4403-a2a8-e1aedba4c96c\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"
Sep 04 22:56:09.818682 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:09.818617    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"
Sep 04 22:56:10.092097 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:10.092070    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"]
Sep 04 22:56:10.094192 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:56:10.094163    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4588074c_7684_4403_a2a8_e1aedba4c96c.slice/crio-96e25b1edf201e67cbf3b98647e318b46ac4d01b24b9c3c7c5769e5809611ffa WatchSource:0}: Error finding container 96e25b1edf201e67cbf3b98647e318b46ac4d01b24b9c3c7c5769e5809611ffa: Status 404 returned error can't find the container with id 96e25b1edf201e67cbf3b98647e318b46ac4d01b24b9c3c7c5769e5809611ffa
Sep 04 22:56:10.225823 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:10.225783    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv" event={"ID":"4588074c-7684-4403-a2a8-e1aedba4c96c","Type":"ContainerStarted","Data":"96e25b1edf201e67cbf3b98647e318b46ac4d01b24b9c3c7c5769e5809611ffa"}
Sep 04 22:56:10.325503 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:10.325466    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:10.418277 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:10.418206    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:12.240334 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:12.240302    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv" event={"ID":"4588074c-7684-4403-a2a8-e1aedba4c96c","Type":"ContainerStarted","Data":"26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a"}
Sep 04 22:56:12.240754 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:12.240413    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"
Sep 04 22:56:18.355101 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:18.355069    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:23.253402 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:23.253370    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"
Sep 04 22:56:23.272845 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:23.272797    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv" podStartSLOduration=13.00904181 podStartE2EDuration="14.272783261s" totalImagesPullingTime="1.263741451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:56:12.258523893 +0000 UTC m=+660.368644482" watchObservedRunningTime="2026-09-04 22:56:23.272783261 +0000 UTC m=+671.382903847"
Sep 04 22:56:24.764806 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:24.764764    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"]
Sep 04 22:56:24.765253 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:24.765000    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv" podUID="4588074c-7684-4403-a2a8-e1aedba4c96c" containerName="limitador" containerID="cri-o://26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a" gracePeriod=30
Sep 04 22:56:25.310386 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.310360    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"
Sep 04 22:56:25.314201 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.314174    2578 generic.go:356] "Generic (PLEG): container finished" podID="4588074c-7684-4403-a2a8-e1aedba4c96c" containerID="26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a" exitCode=0
Sep 04 22:56:25.314311 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.314206    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"
Sep 04 22:56:25.314311 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.314236    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv" event={"ID":"4588074c-7684-4403-a2a8-e1aedba4c96c","Type":"ContainerDied","Data":"26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a"}
Sep 04 22:56:25.314311 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.314279    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-b8dlv" event={"ID":"4588074c-7684-4403-a2a8-e1aedba4c96c","Type":"ContainerDied","Data":"96e25b1edf201e67cbf3b98647e318b46ac4d01b24b9c3c7c5769e5809611ffa"}
Sep 04 22:56:25.314311 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.314303    2578 scope.go:117] "RemoveContainer" containerID="26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a"
Sep 04 22:56:25.322401 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.322378    2578 scope.go:117] "RemoveContainer" containerID="26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a"
Sep 04 22:56:25.322685 ip-10-0-128-32 kubenswrapper[2578]: E0904 22:56:25.322668    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a\": container with ID starting with 26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a not found: ID does not exist" containerID="26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a"
Sep 04 22:56:25.322747 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.322694    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a"} err="failed to get container status \"26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a\": rpc error: code = NotFound desc = could not find container \"26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a\": container with ID starting with 26dd4a70a8a90c128df2db17460d5df8243aeed5c0fbd779969d979c0ca1a90a not found: ID does not exist"
Sep 04 22:56:25.408970 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.408904    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4588074c-7684-4403-a2a8-e1aedba4c96c-config-file\") pod \"4588074c-7684-4403-a2a8-e1aedba4c96c\" (UID: \"4588074c-7684-4403-a2a8-e1aedba4c96c\") "
Sep 04 22:56:25.408970 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.408967    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vddqc\" (UniqueName: \"kubernetes.io/projected/4588074c-7684-4403-a2a8-e1aedba4c96c-kube-api-access-vddqc\") pod \"4588074c-7684-4403-a2a8-e1aedba4c96c\" (UID: \"4588074c-7684-4403-a2a8-e1aedba4c96c\") "
Sep 04 22:56:25.409208 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.409188    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4588074c-7684-4403-a2a8-e1aedba4c96c-config-file" (OuterVolumeSpecName: "config-file") pod "4588074c-7684-4403-a2a8-e1aedba4c96c" (UID: "4588074c-7684-4403-a2a8-e1aedba4c96c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:56:25.410957 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.410931    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4588074c-7684-4403-a2a8-e1aedba4c96c-kube-api-access-vddqc" (OuterVolumeSpecName: "kube-api-access-vddqc") pod "4588074c-7684-4403-a2a8-e1aedba4c96c" (UID: "4588074c-7684-4403-a2a8-e1aedba4c96c"). InnerVolumeSpecName "kube-api-access-vddqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:56:25.510011 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.509982    2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4588074c-7684-4403-a2a8-e1aedba4c96c-config-file\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:56:25.510011 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.510007    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vddqc\" (UniqueName: \"kubernetes.io/projected/4588074c-7684-4403-a2a8-e1aedba4c96c-kube-api-access-vddqc\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 22:56:25.638526 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.638482    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"]
Sep 04 22:56:25.641814 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:25.641788    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-b8dlv"]
Sep 04 22:56:26.418694 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:26.418661    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4588074c-7684-4403-a2a8-e1aedba4c96c" path="/var/lib/kubelet/pods/4588074c-7684-4403-a2a8-e1aedba4c96c/volumes"
Sep 04 22:56:26.773375 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:26.773290    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:26.871395 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:26.871356    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:27.258892 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.258856    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"]
Sep 04 22:56:27.259257 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.259244    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4588074c-7684-4403-a2a8-e1aedba4c96c" containerName="limitador"
Sep 04 22:56:27.259311 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.259261    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4588074c-7684-4403-a2a8-e1aedba4c96c" containerName="limitador"
Sep 04 22:56:27.259344 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.259319    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4588074c-7684-4403-a2a8-e1aedba4c96c" containerName="limitador"
Sep 04 22:56:27.263420 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.263397    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"
Sep 04 22:56:27.265970 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.265941    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 04 22:56:27.266117 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.266096    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 04 22:56:27.266793 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.266777    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 22:56:27.266857 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.266777    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-qcxzd\""
Sep 04 22:56:27.270823 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.270678    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"]
Sep 04 22:56:27.281236 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.281211    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:27.324098 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.324076    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e94f394a-ce5c-4963-8d87-1b23d7b245fc-data\") pod \"postgres-69fb4594cf-v4p9p\" (UID: \"e94f394a-ce5c-4963-8d87-1b23d7b245fc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"
Sep 04 22:56:27.324228 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.324113    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj4sk\" (UniqueName: \"kubernetes.io/projected/e94f394a-ce5c-4963-8d87-1b23d7b245fc-kube-api-access-tj4sk\") pod \"postgres-69fb4594cf-v4p9p\" (UID: \"e94f394a-ce5c-4963-8d87-1b23d7b245fc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"
Sep 04 22:56:27.424732 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.424696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e94f394a-ce5c-4963-8d87-1b23d7b245fc-data\") pod \"postgres-69fb4594cf-v4p9p\" (UID: \"e94f394a-ce5c-4963-8d87-1b23d7b245fc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"
Sep 04 22:56:27.425223 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.424742    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj4sk\" (UniqueName: \"kubernetes.io/projected/e94f394a-ce5c-4963-8d87-1b23d7b245fc-kube-api-access-tj4sk\") pod \"postgres-69fb4594cf-v4p9p\" (UID: \"e94f394a-ce5c-4963-8d87-1b23d7b245fc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"
Sep 04 22:56:27.425223 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.425151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e94f394a-ce5c-4963-8d87-1b23d7b245fc-data\") pod \"postgres-69fb4594cf-v4p9p\" (UID: \"e94f394a-ce5c-4963-8d87-1b23d7b245fc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"
Sep 04 22:56:27.436165 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.436132    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj4sk\" (UniqueName: \"kubernetes.io/projected/e94f394a-ce5c-4963-8d87-1b23d7b245fc-kube-api-access-tj4sk\") pod \"postgres-69fb4594cf-v4p9p\" (UID: \"e94f394a-ce5c-4963-8d87-1b23d7b245fc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"
Sep 04 22:56:27.574750 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.574719    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"
Sep 04 22:56:27.844198 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:27.844169    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"]
Sep 04 22:56:27.845983 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:56:27.845959    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94f394a_ce5c_4963_8d87_1b23d7b245fc.slice/crio-d5cadc33990ae20d88dbee50df88b99b10a68c4ff0c699883738f51ba387f5fa WatchSource:0}: Error finding container d5cadc33990ae20d88dbee50df88b99b10a68c4ff0c699883738f51ba387f5fa: Status 404 returned error can't find the container with id d5cadc33990ae20d88dbee50df88b99b10a68c4ff0c699883738f51ba387f5fa
Sep 04 22:56:28.333342 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:28.333307    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p" event={"ID":"e94f394a-ce5c-4963-8d87-1b23d7b245fc","Type":"ContainerStarted","Data":"d5cadc33990ae20d88dbee50df88b99b10a68c4ff0c699883738f51ba387f5fa"}
Sep 04 22:56:33.371530 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:33.371485    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p" event={"ID":"e94f394a-ce5c-4963-8d87-1b23d7b245fc","Type":"ContainerStarted","Data":"28d0bf160e2918b5a1f0b5e615fe4249a3d0f8aeb07b917c3399ba101216242e"}
Sep 04 22:56:33.371909 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:33.371617    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"
Sep 04 22:56:33.387755 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:33.387701    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p" podStartSLOduration=1.501903339 podStartE2EDuration="6.387684038s" totalImagesPullingTime="4.885780699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:56:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:56:33.386487951 +0000 UTC m=+681.496608540" watchObservedRunningTime="2026-09-04 22:56:33.387684038 +0000 UTC m=+681.497804625"
Sep 04 22:56:39.405374 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:39.405344    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v4p9p"
Sep 04 22:56:39.727524 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:39.727440    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:40.205836 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:40.205793    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:46.704979 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:46.704941    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:46.784955 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:46.784926    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:46.849725 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:46.849681    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:46.925858 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:46.925824    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:50.215994 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:50.215958    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:56:50.316701 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:56:50.316666    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:19.938212 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:19.938178    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:20.039213 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:20.039177    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:21.434023 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:21.433986    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:21.532549 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:21.532498    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:21.685364 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:21.685266    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:21.918286 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:21.918230    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:22.022622 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.022504    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:22.119229 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.119198    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:22.156981 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.156940    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"]
Sep 04 22:57:22.161634 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.161603    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"
Sep 04 22:57:22.164183 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.164160    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4bwmg\""
Sep 04 22:57:22.164293 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.164178    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 04 22:57:22.171252 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.171230    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"]
Sep 04 22:57:22.182157 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.182134    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"]
Sep 04 22:57:22.185989 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.185971    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"
Sep 04 22:57:22.206197 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.200794    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7mqv\" (UniqueName: \"kubernetes.io/projected/3646b8ff-9710-4dfd-8702-32d8e3ca23e3-kube-api-access-r7mqv\") pod \"payload-pre-processing-59fbc47bf8-6vxmn\" (UID: \"3646b8ff-9710-4dfd-8702-32d8e3ca23e3\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"
Sep 04 22:57:22.206197 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.200879    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3646b8ff-9710-4dfd-8702-32d8e3ca23e3-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-6vxmn\" (UID: \"3646b8ff-9710-4dfd-8702-32d8e3ca23e3\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"
Sep 04 22:57:22.206197 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.201978    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"]
Sep 04 22:57:22.206652 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.206451    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:22.301716 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.301607    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7mqv\" (UniqueName: \"kubernetes.io/projected/3646b8ff-9710-4dfd-8702-32d8e3ca23e3-kube-api-access-r7mqv\") pod \"payload-pre-processing-59fbc47bf8-6vxmn\" (UID: \"3646b8ff-9710-4dfd-8702-32d8e3ca23e3\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"
Sep 04 22:57:22.301716 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.301680    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3646b8ff-9710-4dfd-8702-32d8e3ca23e3-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-6vxmn\" (UID: \"3646b8ff-9710-4dfd-8702-32d8e3ca23e3\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"
Sep 04 22:57:22.301928 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.301763    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw524\" (UniqueName: \"kubernetes.io/projected/337ff73a-57c3-4d64-a0a6-e942e76477d5-kube-api-access-pw524\") pod \"payload-processing-5b54b7fbb7-2pxrz\" (UID: \"337ff73a-57c3-4d64-a0a6-e942e76477d5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"
Sep 04 22:57:22.301928 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.301816    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337ff73a-57c3-4d64-a0a6-e942e76477d5-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-2pxrz\" (UID: \"337ff73a-57c3-4d64-a0a6-e942e76477d5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"
Sep 04 22:57:22.302317 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.302296    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3646b8ff-9710-4dfd-8702-32d8e3ca23e3-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-6vxmn\" (UID: \"3646b8ff-9710-4dfd-8702-32d8e3ca23e3\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"
Sep 04 22:57:22.310063 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.310037    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7mqv\" (UniqueName: \"kubernetes.io/projected/3646b8ff-9710-4dfd-8702-32d8e3ca23e3-kube-api-access-r7mqv\") pod \"payload-pre-processing-59fbc47bf8-6vxmn\" (UID: \"3646b8ff-9710-4dfd-8702-32d8e3ca23e3\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"
Sep 04 22:57:22.403095 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.403059    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337ff73a-57c3-4d64-a0a6-e942e76477d5-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-2pxrz\" (UID: \"337ff73a-57c3-4d64-a0a6-e942e76477d5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"
Sep 04 22:57:22.403271 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.403147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw524\" (UniqueName: \"kubernetes.io/projected/337ff73a-57c3-4d64-a0a6-e942e76477d5-kube-api-access-pw524\") pod \"payload-processing-5b54b7fbb7-2pxrz\" (UID: \"337ff73a-57c3-4d64-a0a6-e942e76477d5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"
Sep 04 22:57:22.403785 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.403763    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337ff73a-57c3-4d64-a0a6-e942e76477d5-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-2pxrz\" (UID: \"337ff73a-57c3-4d64-a0a6-e942e76477d5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"
Sep 04 22:57:22.412566 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.412543    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw524\" (UniqueName: \"kubernetes.io/projected/337ff73a-57c3-4d64-a0a6-e942e76477d5-kube-api-access-pw524\") pod \"payload-processing-5b54b7fbb7-2pxrz\" (UID: \"337ff73a-57c3-4d64-a0a6-e942e76477d5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"
Sep 04 22:57:22.472226 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.472194    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"
Sep 04 22:57:22.496463 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.496436    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"
Sep 04 22:57:22.747789 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.747748    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"]
Sep 04 22:57:22.749422 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:57:22.749387    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3646b8ff_9710_4dfd_8702_32d8e3ca23e3.slice/crio-0c44a9f0cc66ca2a64dafc769a5d6b6d19d63ea6e26623ce4972f58921e6a62b WatchSource:0}: Error finding container 0c44a9f0cc66ca2a64dafc769a5d6b6d19d63ea6e26623ce4972f58921e6a62b: Status 404 returned error can't find the container with id 0c44a9f0cc66ca2a64dafc769a5d6b6d19d63ea6e26623ce4972f58921e6a62b
Sep 04 22:57:22.751927 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.751906    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 22:57:22.783121 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:22.783091    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"]
Sep 04 22:57:22.784488 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:57:22.784466    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337ff73a_57c3_4d64_a0a6_e942e76477d5.slice/crio-552290082d4bb78dca226b6d1eb5d8b91c47600d0680b454fc347e4520d5ad21 WatchSource:0}: Error finding container 552290082d4bb78dca226b6d1eb5d8b91c47600d0680b454fc347e4520d5ad21: Status 404 returned error can't find the container with id 552290082d4bb78dca226b6d1eb5d8b91c47600d0680b454fc347e4520d5ad21
Sep 04 22:57:23.647705 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:23.647634    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz" event={"ID":"337ff73a-57c3-4d64-a0a6-e942e76477d5","Type":"ContainerStarted","Data":"552290082d4bb78dca226b6d1eb5d8b91c47600d0680b454fc347e4520d5ad21"}
Sep 04 22:57:23.649682 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:23.649631    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn" event={"ID":"3646b8ff-9710-4dfd-8702-32d8e3ca23e3","Type":"ContainerStarted","Data":"0c44a9f0cc66ca2a64dafc769a5d6b6d19d63ea6e26623ce4972f58921e6a62b"}
Sep 04 22:57:24.657550 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:24.657509    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz" event={"ID":"337ff73a-57c3-4d64-a0a6-e942e76477d5","Type":"ContainerStarted","Data":"b155356bbc45e35ec417eddf55c0722e61e0413e696a7b8362674006a4eb64b4"}
Sep 04 22:57:24.658013 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:24.657736    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"
Sep 04 22:57:24.659721 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:24.659691    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn" event={"ID":"3646b8ff-9710-4dfd-8702-32d8e3ca23e3","Type":"ContainerStarted","Data":"0685f167b90243bb1c424a65322ce12e5d6017e3f6f85c9c036af764feec4df8"}
Sep 04 22:57:24.659862 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:24.659781    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"
Sep 04 22:57:24.673896 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:24.673849    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz" podStartSLOduration=0.913799226 podStartE2EDuration="2.673833146s" totalImagesPullingTime="1.76003392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:57:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:57:24.673224568 +0000 UTC m=+732.783345159" watchObservedRunningTime="2026-09-04 22:57:24.673833146 +0000 UTC m=+732.783953732"
Sep 04 22:57:31.685780 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:31.685745    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:31.788212 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:31.788153    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:35.667765 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:35.667728    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn"
Sep 04 22:57:35.668159 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:35.667802    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pxrz"
Sep 04 22:57:35.685358 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:35.685307    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-6vxmn" podStartSLOduration=11.884460741 podStartE2EDuration="13.685293807s" totalImagesPullingTime="1.800833066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:57:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:57:24.688504213 +0000 UTC m=+732.798624835" watchObservedRunningTime="2026-09-04 22:57:35.685293807 +0000 UTC m=+743.795414442"
Sep 04 22:57:39.725528 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:39.725493    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:47.492587 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.492549    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"]
Sep 04 22:57:47.496206 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.496180    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.499719 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.499697    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 04 22:57:47.499838 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.499746    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vktld\""
Sep 04 22:57:47.499838 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.499718    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 04 22:57:47.499838 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.499810    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 04 22:57:47.506078 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.506055    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"]
Sep 04 22:57:47.524864 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.524838    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:47.526079 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.526058    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.526193 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.526090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.526193 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.526120    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.526265 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.526193    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.526265 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.526227    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.526265 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.526261    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5czlw\" (UniqueName: \"kubernetes.io/projected/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-kube-api-access-5czlw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.627500 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.627462    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.627500 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.627503    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.627771 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.627541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czlw\" (UniqueName: \"kubernetes.io/projected/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-kube-api-access-5czlw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.627771 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.627593    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.627771 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.627621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.627771 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.627680    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.627992 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.627961    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.628049 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.628032    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.628203 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.628181    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.629984 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.629960    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.630150 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.630134    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.639185 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.639162    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5czlw\" (UniqueName: \"kubernetes.io/projected/5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb-kube-api-access-5czlw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zw5tn\" (UID: \"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:47.810789 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:47.810695    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:48.094863 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:48.094831    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"]
Sep 04 22:57:48.096188 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:57:48.096160    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f2811b7_ab3e_4c93_ac4e_0f0585c5b2fb.slice/crio-94376944b7a288abcb13f87de45ec1fcdd933eef248f20ba1bebf60cfb7c4b46 WatchSource:0}: Error finding container 94376944b7a288abcb13f87de45ec1fcdd933eef248f20ba1bebf60cfb7c4b46: Status 404 returned error can't find the container with id 94376944b7a288abcb13f87de45ec1fcdd933eef248f20ba1bebf60cfb7c4b46
Sep 04 22:57:48.799902 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:48.799866    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn" event={"ID":"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb","Type":"ContainerStarted","Data":"94376944b7a288abcb13f87de45ec1fcdd933eef248f20ba1bebf60cfb7c4b46"}
Sep 04 22:57:49.807583 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:49.807543    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn" event={"ID":"5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb","Type":"ContainerStarted","Data":"0ba770955a02680c9e4ccd9ea6dc7049ef7dad02f6f9568a629e89b2de628dfe"}
Sep 04 22:57:49.826653 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:49.826585    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn" podStartSLOduration=1.8885905539999999 podStartE2EDuration="2.826569637s" totalImagesPullingTime="937.979083ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:57:49.823528506 +0000 UTC m=+757.933649091" watchObservedRunningTime="2026-09-04 22:57:49.826569637 +0000 UTC m=+757.936690222"
Sep 04 22:57:50.808122 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:50.808089    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:57.714151 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:57.714120    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:57:57.811353 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:57.811310    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:57.811522 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:57.811386    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:57.824503 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:57.824478    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:57:57.862587 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:57:57.862556    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zw5tn"
Sep 04 22:58:04.925909 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:04.925876    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:58:10.745849 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:10.745812    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:58:12.891313 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:12.891278    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"]
Sep 04 22:58:12.894445 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:12.894421    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:12.896833 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:12.896812    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 04 22:58:12.903545 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:12.903520    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"]
Sep 04 22:58:12.918438 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:12.918414    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:58:12.953941 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:12.953912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8efc8e75-8870-45a6-98a0-29c5ef0d8426-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:12.954098 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:12.953948    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8efc8e75-8870-45a6-98a0-29c5ef0d8426-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:12.954098 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:12.954007    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8efc8e75-8870-45a6-98a0-29c5ef0d8426-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:12.954098 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:12.954035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr6dh\" (UniqueName: \"kubernetes.io/projected/8efc8e75-8870-45a6-98a0-29c5ef0d8426-kube-api-access-dr6dh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:12.954264 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:12.954109    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8efc8e75-8870-45a6-98a0-29c5ef0d8426-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:12.954264 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:12.954158    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8efc8e75-8870-45a6-98a0-29c5ef0d8426-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.055252 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.055216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8efc8e75-8870-45a6-98a0-29c5ef0d8426-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.055450 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.055259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8efc8e75-8870-45a6-98a0-29c5ef0d8426-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.055450 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.055303    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8efc8e75-8870-45a6-98a0-29c5ef0d8426-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.055450 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.055326    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8efc8e75-8870-45a6-98a0-29c5ef0d8426-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.055450 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.055358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8efc8e75-8870-45a6-98a0-29c5ef0d8426-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.055680 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.055503    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr6dh\" (UniqueName: \"kubernetes.io/projected/8efc8e75-8870-45a6-98a0-29c5ef0d8426-kube-api-access-dr6dh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.055750 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.055694    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8efc8e75-8870-45a6-98a0-29c5ef0d8426-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.055809 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.055791    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8efc8e75-8870-45a6-98a0-29c5ef0d8426-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.055857 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.055818    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8efc8e75-8870-45a6-98a0-29c5ef0d8426-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.057725 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.057689    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8efc8e75-8870-45a6-98a0-29c5ef0d8426-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.057852 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.057807    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8efc8e75-8870-45a6-98a0-29c5ef0d8426-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.063795 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.063773    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr6dh\" (UniqueName: \"kubernetes.io/projected/8efc8e75-8870-45a6-98a0-29c5ef0d8426-kube-api-access-dr6dh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5xj7\" (UID: \"8efc8e75-8870-45a6-98a0-29c5ef0d8426\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.205677 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.205571    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:13.477887 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.477858    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"]
Sep 04 22:58:13.479097 ip-10-0-128-32 kubenswrapper[2578]: W0904 22:58:13.479071    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8efc8e75_8870_45a6_98a0_29c5ef0d8426.slice/crio-6d3c10d5e11880f0a8bb6657e8f8d6fa6cc025091451fe0cac7e8063fdff365b WatchSource:0}: Error finding container 6d3c10d5e11880f0a8bb6657e8f8d6fa6cc025091451fe0cac7e8063fdff365b: Status 404 returned error can't find the container with id 6d3c10d5e11880f0a8bb6657e8f8d6fa6cc025091451fe0cac7e8063fdff365b
Sep 04 22:58:13.942258 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.942213    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7" event={"ID":"8efc8e75-8870-45a6-98a0-29c5ef0d8426","Type":"ContainerStarted","Data":"7425c3c87e6b034701bb8ed94e839f43e2b584046376e4feb1200c26969a3cf7"}
Sep 04 22:58:13.942682 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.942313    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7" event={"ID":"8efc8e75-8870-45a6-98a0-29c5ef0d8426","Type":"ContainerStarted","Data":"6d3c10d5e11880f0a8bb6657e8f8d6fa6cc025091451fe0cac7e8063fdff365b"}
Sep 04 22:58:13.961901 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:13.961849    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7" podStartSLOduration=1.804713681 podStartE2EDuration="1.961833198s" totalImagesPullingTime="157.119517ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:58:13.95980913 +0000 UTC m=+782.069929991" watchObservedRunningTime="2026-09-04 22:58:13.961833198 +0000 UTC m=+782.071953783"
Sep 04 22:58:18.326596 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:18.326562    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:58:18.387552 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:18.387519    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:58:22.619615 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:22.619579    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:58:23.206265 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:23.206225    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:23.206455 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:23.206288    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:23.218730 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:23.218701    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:58:24.007880 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:58:24.007848    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5xj7"
Sep 04 22:59:48.628194 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:59:48.628155    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 22:59:50.020516 ip-10-0-128-32 kubenswrapper[2578]: I0904 22:59:50.020484    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:00.156844 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:00.156807    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809380-p5r44"]
Sep 04 23:00:00.160350 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:00.160332    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809380-p5r44"
Sep 04 23:00:00.164163 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:00.164140    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:00:00.169872 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:00.169842    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809380-p5r44"]
Sep 04 23:00:00.193659 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:00.193621    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809380-p5r44"
Sep 04 23:00:00.208780 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:00.208750    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:00.302260 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:00.302212    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:00.471678 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:00.471635    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809380-p5r44"]
Sep 04 23:00:00.473816 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:00:00.473791    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35cffcac_be02_4cd8_9730_f76b35cf5f4c.slice/crio-a0d4b142ee12da01d3ffafce7d90f0a05404318d7a61835d431420a698b5d5a5 WatchSource:0}: Error finding container a0d4b142ee12da01d3ffafce7d90f0a05404318d7a61835d431420a698b5d5a5: Status 404 returned error can't find the container with id a0d4b142ee12da01d3ffafce7d90f0a05404318d7a61835d431420a698b5d5a5
Sep 04 23:00:00.531896 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:00.531864    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809380-p5r44" event={"ID":"35cffcac-be02-4cd8-9730-f76b35cf5f4c","Type":"ContainerStarted","Data":"a0d4b142ee12da01d3ffafce7d90f0a05404318d7a61835d431420a698b5d5a5"}
Sep 04 23:00:01.620319 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:01.620280    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:01.721272 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:01.721234    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:03.552819 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:03.552787    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809380-p5r44" event={"ID":"35cffcac-be02-4cd8-9730-f76b35cf5f4c","Type":"ContainerStarted","Data":"f2f8e421f8041c20d99c48bcccdc1a5aa2c7fbc191dcfb59a6d1c8c5626f1d6a"}
Sep 04 23:00:03.570391 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:03.570326    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809380-p5r44" podStartSLOduration=0.822652865 podStartE2EDuration="3.570308119s" totalImagesPullingTime="2.747655254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:03.568201703 +0000 UTC m=+891.678322292" watchObservedRunningTime="2026-09-04 23:00:03.570308119 +0000 UTC m=+891.680428745"
Sep 04 23:00:04.559424 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:04.559265    2578 generic.go:356] "Generic (PLEG): container finished" podID="35cffcac-be02-4cd8-9730-f76b35cf5f4c" containerID="f2f8e421f8041c20d99c48bcccdc1a5aa2c7fbc191dcfb59a6d1c8c5626f1d6a" exitCode=0
Sep 04 23:00:04.559424 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:04.559350    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809380-p5r44" event={"ID":"35cffcac-be02-4cd8-9730-f76b35cf5f4c","Type":"ContainerDied","Data":"f2f8e421f8041c20d99c48bcccdc1a5aa2c7fbc191dcfb59a6d1c8c5626f1d6a"}
Sep 04 23:00:05.698113 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:05.698085    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809380-p5r44"
Sep 04 23:00:06.571514 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:06.571481    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809380-p5r44"
Sep 04 23:00:06.571514 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:06.571505    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809380-p5r44" event={"ID":"35cffcac-be02-4cd8-9730-f76b35cf5f4c","Type":"ContainerDied","Data":"a0d4b142ee12da01d3ffafce7d90f0a05404318d7a61835d431420a698b5d5a5"}
Sep 04 23:00:06.571774 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:06.571548    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0d4b142ee12da01d3ffafce7d90f0a05404318d7a61835d431420a698b5d5a5"
Sep 04 23:00:12.386902 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:12.386870    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 23:00:12.388475 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:12.388453    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 23:00:17.425030 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:17.424992    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:17.809456 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:17.809357    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:17.903376 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:17.903333    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:18.130054 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:18.130020    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:18.897154 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:18.897123    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:18.996165 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:18.996110    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:19.652293 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:19.652257    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:20.631450 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:20.631409    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:20.728379 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:20.728345    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:21.772488 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:21.772457    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:21.840990 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:21.840955    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:21.959358 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:21.959327    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:22.306881 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.306847    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:22.401161 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.401126    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:22.420047 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.420020    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"]
Sep 04 23:00:22.420531 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.420494    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35cffcac-be02-4cd8-9730-f76b35cf5f4c" containerName="cleanup"
Sep 04 23:00:22.420531 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.420516    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="35cffcac-be02-4cd8-9730-f76b35cf5f4c" containerName="cleanup"
Sep 04 23:00:22.420713 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.420618    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="35cffcac-be02-4cd8-9730-f76b35cf5f4c" containerName="cleanup"
Sep 04 23:00:22.426546 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.426527    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"
Sep 04 23:00:22.428964 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.428941    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-a64414\""
Sep 04 23:00:22.429820 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.429795    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-a64414-dockercfg-wmbks\""
Sep 04 23:00:22.432286 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.432262    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"]
Sep 04 23:00:22.441017 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.440996    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"]
Sep 04 23:00:22.444824 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.444807    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"
Sep 04 23:00:22.455553 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.455534    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"]
Sep 04 23:00:22.508630 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.508600    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d8fbd59-9084-4693-a2c0-1b1ba5d43185-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp\" (UID: \"8d8fbd59-9084-4693-a2c0-1b1ba5d43185\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"
Sep 04 23:00:22.508772 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.508703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zhgw\" (UniqueName: \"kubernetes.io/projected/40c1e521-9ffa-4dc1-ac59-1e4e9226202e-kube-api-access-4zhgw\") pod \"payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4\" (UID: \"40c1e521-9ffa-4dc1-ac59-1e4e9226202e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"
Sep 04 23:00:22.508831 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.508772    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzn6n\" (UniqueName: \"kubernetes.io/projected/8d8fbd59-9084-4693-a2c0-1b1ba5d43185-kube-api-access-xzn6n\") pod \"payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp\" (UID: \"8d8fbd59-9084-4693-a2c0-1b1ba5d43185\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"
Sep 04 23:00:22.508989 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.508971    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40c1e521-9ffa-4dc1-ac59-1e4e9226202e-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4\" (UID: \"40c1e521-9ffa-4dc1-ac59-1e4e9226202e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"
Sep 04 23:00:22.610291 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.610264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40c1e521-9ffa-4dc1-ac59-1e4e9226202e-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4\" (UID: \"40c1e521-9ffa-4dc1-ac59-1e4e9226202e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"
Sep 04 23:00:22.610454 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.610361    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d8fbd59-9084-4693-a2c0-1b1ba5d43185-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp\" (UID: \"8d8fbd59-9084-4693-a2c0-1b1ba5d43185\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"
Sep 04 23:00:22.610454 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.610408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zhgw\" (UniqueName: \"kubernetes.io/projected/40c1e521-9ffa-4dc1-ac59-1e4e9226202e-kube-api-access-4zhgw\") pod \"payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4\" (UID: \"40c1e521-9ffa-4dc1-ac59-1e4e9226202e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"
Sep 04 23:00:22.610566 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.610455    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzn6n\" (UniqueName: \"kubernetes.io/projected/8d8fbd59-9084-4693-a2c0-1b1ba5d43185-kube-api-access-xzn6n\") pod \"payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp\" (UID: \"8d8fbd59-9084-4693-a2c0-1b1ba5d43185\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"
Sep 04 23:00:22.611080 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.611060    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40c1e521-9ffa-4dc1-ac59-1e4e9226202e-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4\" (UID: \"40c1e521-9ffa-4dc1-ac59-1e4e9226202e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"
Sep 04 23:00:22.611192 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.611172    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d8fbd59-9084-4693-a2c0-1b1ba5d43185-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp\" (UID: \"8d8fbd59-9084-4693-a2c0-1b1ba5d43185\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"
Sep 04 23:00:22.618410 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.618380    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzn6n\" (UniqueName: \"kubernetes.io/projected/8d8fbd59-9084-4693-a2c0-1b1ba5d43185-kube-api-access-xzn6n\") pod \"payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp\" (UID: \"8d8fbd59-9084-4693-a2c0-1b1ba5d43185\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"
Sep 04 23:00:22.618530 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.618433    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zhgw\" (UniqueName: \"kubernetes.io/projected/40c1e521-9ffa-4dc1-ac59-1e4e9226202e-kube-api-access-4zhgw\") pod \"payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4\" (UID: \"40c1e521-9ffa-4dc1-ac59-1e4e9226202e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"
Sep 04 23:00:22.740415 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.740376    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"
Sep 04 23:00:22.755765 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:22.755737    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"
Sep 04 23:00:23.023859 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:23.023680    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"]
Sep 04 23:00:23.036004 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:23.035981    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"]
Sep 04 23:00:23.037310 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:00:23.037285    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c1e521_9ffa_4dc1_ac59_1e4e9226202e.slice/crio-d36fd63b6d5ee829118c11f0a5c8322c74f9ddb1e3e9baf39c11b0ad6ffdb661 WatchSource:0}: Error finding container d36fd63b6d5ee829118c11f0a5c8322c74f9ddb1e3e9baf39c11b0ad6ffdb661: Status 404 returned error can't find the container with id d36fd63b6d5ee829118c11f0a5c8322c74f9ddb1e3e9baf39c11b0ad6ffdb661
Sep 04 23:00:23.672854 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:23.672814    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp" event={"ID":"8d8fbd59-9084-4693-a2c0-1b1ba5d43185","Type":"ContainerStarted","Data":"965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8"}
Sep 04 23:00:23.673052 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:23.672885    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp" event={"ID":"8d8fbd59-9084-4693-a2c0-1b1ba5d43185","Type":"ContainerStarted","Data":"e5b9e208e57c5a47b94e5b70662bae329e30b0f38a34801dfee3f1678eb2be09"}
Sep 04 23:00:23.673052 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:23.672923    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"
Sep 04 23:00:23.674741 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:23.674708    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4" event={"ID":"40c1e521-9ffa-4dc1-ac59-1e4e9226202e","Type":"ContainerStarted","Data":"f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2"}
Sep 04 23:00:23.674891 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:23.674763    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4" event={"ID":"40c1e521-9ffa-4dc1-ac59-1e4e9226202e","Type":"ContainerStarted","Data":"d36fd63b6d5ee829118c11f0a5c8322c74f9ddb1e3e9baf39c11b0ad6ffdb661"}
Sep 04 23:00:23.674891 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:23.674859    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"
Sep 04 23:00:23.688077 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:23.688020    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp" podStartSLOduration=1.688001031 podStartE2EDuration="1.688001031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:23.686850707 +0000 UTC m=+911.796971294" watchObservedRunningTime="2026-09-04 23:00:23.688001031 +0000 UTC m=+911.798121617"
Sep 04 23:00:23.703671 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:23.703612    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4" podStartSLOduration=1.70359752 podStartE2EDuration="1.70359752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:23.701486029 +0000 UTC m=+911.811606619" watchObservedRunningTime="2026-09-04 23:00:23.70359752 +0000 UTC m=+911.813718106"
Sep 04 23:00:25.184255 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.184219    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:25.284537 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.284494    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:25.458111 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.458018    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:25.695445 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.695408    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"]
Sep 04 23:00:25.699747 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.699725    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.702269 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.702250    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-921d8a5d-openshift-default-dockercfg-l6f69\""
Sep 04 23:00:25.711117 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.711065    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"]
Sep 04 23:00:25.750126 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.750082    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:25.839409 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.839362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-podinfo\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.839593 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.839465    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74d699b2-c57d-4386-88fe-794b2ff5fd90-istiod-ca-cert\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.839593 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.839518    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-workload-certs\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.839593 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.839545    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfhlc\" (UniqueName: \"kubernetes.io/projected/74d699b2-c57d-4386-88fe-794b2ff5fd90-kube-api-access-pfhlc\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.839593 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.839581    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-workload-socket\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.839841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.839676    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-data\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.839841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.839731    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-credential-socket\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.839841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.839770    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-token\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.839841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.839800    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-envoy\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.909989 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.909960    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:25.940345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.940312    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74d699b2-c57d-4386-88fe-794b2ff5fd90-istiod-ca-cert\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.940513 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.940374    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-workload-certs\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.940513 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.940406    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfhlc\" (UniqueName: \"kubernetes.io/projected/74d699b2-c57d-4386-88fe-794b2ff5fd90-kube-api-access-pfhlc\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.940513 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.940437    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-workload-socket\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.940513 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.940463    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-data\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.940762 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.940526    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-credential-socket\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.940762 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.940563    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-token\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.940762 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.940603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-envoy\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.940762 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.940662    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-podinfo\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.940987 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.940874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-workload-socket\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.941125 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.941096    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-data\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.941258 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.941159    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-credential-socket\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.941258 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.941169    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-workload-certs\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.941529 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.941508    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74d699b2-c57d-4386-88fe-794b2ff5fd90-istiod-ca-cert\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.943224 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.943192    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-envoy\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.943506 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.943488    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-podinfo\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.948412 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.948391    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfhlc\" (UniqueName: \"kubernetes.io/projected/74d699b2-c57d-4386-88fe-794b2ff5fd90-kube-api-access-pfhlc\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:25.948522 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:25.948411    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-token\") pod \"e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") " pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:26.011468 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:26.011410    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:26.097209 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:26.097183    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:26.290362 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:26.290330    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"]
Sep 04 23:00:26.291817 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:00:26.291780    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74d699b2_c57d_4386_88fe_794b2ff5fd90.slice/crio-fb0d71009ea8762477473d675603fa71e00caf84fc14904b77e5f8e1bb66ed3a WatchSource:0}: Error finding container fb0d71009ea8762477473d675603fa71e00caf84fc14904b77e5f8e1bb66ed3a: Status 404 returned error can't find the container with id fb0d71009ea8762477473d675603fa71e00caf84fc14904b77e5f8e1bb66ed3a
Sep 04 23:00:26.695853 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:26.695818    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" event={"ID":"74d699b2-c57d-4386-88fe-794b2ff5fd90","Type":"ContainerStarted","Data":"fb0d71009ea8762477473d675603fa71e00caf84fc14904b77e5f8e1bb66ed3a"}
Sep 04 23:00:27.478529 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:27.478476    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:27.803898 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:27.803822    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:28.364182 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:28.364148    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:33.680932 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:33.680895    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:33.777766 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:33.777734    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:34.088084 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:34.088050    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:34.193974 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:34.193938    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:34.452566 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:34.452476    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:34.683256 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:34.683223    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"
Sep 04 23:00:34.683696 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:34.683300    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"
Sep 04 23:00:34.849334 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:34.849301    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:34.949376 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:34.949344    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:35.527139 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.527108    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"]
Sep 04 23:00:35.531970 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.531947    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"
Sep 04 23:00:35.534428 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.534390    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-63d928-dockercfg-7th8p\""
Sep 04 23:00:35.534612 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.534597    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-63d928\""
Sep 04 23:00:35.540881 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.540854    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"]
Sep 04 23:00:35.556472 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.556441    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"]
Sep 04 23:00:35.560927 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.560907    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"
Sep 04 23:00:35.573040 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.573017    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"]
Sep 04 23:00:35.601346 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.601320    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:35.631775 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.631747    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pjhr\" (UniqueName: \"kubernetes.io/projected/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e-kube-api-access-5pjhr\") pod \"payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr\" (UID: \"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"
Sep 04 23:00:35.631941 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.631905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr\" (UID: \"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"
Sep 04 23:00:35.663046 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.663015    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:35.733016 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.732977    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pjhr\" (UniqueName: \"kubernetes.io/projected/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e-kube-api-access-5pjhr\") pod \"payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr\" (UID: \"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"
Sep 04 23:00:35.733464 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.733132    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a343ffe9-9686-449d-8e68-195643ea9565-plugins-config-volume\") pod \"payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn\" (UID: \"a343ffe9-9686-449d-8e68-195643ea9565\") " pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"
Sep 04 23:00:35.733464 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.733245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr\" (UID: \"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"
Sep 04 23:00:35.733464 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.733283    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhjsg\" (UniqueName: \"kubernetes.io/projected/a343ffe9-9686-449d-8e68-195643ea9565-kube-api-access-fhjsg\") pod \"payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn\" (UID: \"a343ffe9-9686-449d-8e68-195643ea9565\") " pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"
Sep 04 23:00:35.734168 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.734140    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr\" (UID: \"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"
Sep 04 23:00:35.743801 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.743776    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pjhr\" (UniqueName: \"kubernetes.io/projected/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e-kube-api-access-5pjhr\") pod \"payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr\" (UID: \"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"
Sep 04 23:00:35.833927 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.833889    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjsg\" (UniqueName: \"kubernetes.io/projected/a343ffe9-9686-449d-8e68-195643ea9565-kube-api-access-fhjsg\") pod \"payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn\" (UID: \"a343ffe9-9686-449d-8e68-195643ea9565\") " pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"
Sep 04 23:00:35.834091 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.833996    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a343ffe9-9686-449d-8e68-195643ea9565-plugins-config-volume\") pod \"payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn\" (UID: \"a343ffe9-9686-449d-8e68-195643ea9565\") " pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"
Sep 04 23:00:35.834918 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.834892    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a343ffe9-9686-449d-8e68-195643ea9565-plugins-config-volume\") pod \"payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn\" (UID: \"a343ffe9-9686-449d-8e68-195643ea9565\") " pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"
Sep 04 23:00:35.841688 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.841664    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhjsg\" (UniqueName: \"kubernetes.io/projected/a343ffe9-9686-449d-8e68-195643ea9565-kube-api-access-fhjsg\") pod \"payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn\" (UID: \"a343ffe9-9686-449d-8e68-195643ea9565\") " pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"
Sep 04 23:00:35.844495 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.844461    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"
Sep 04 23:00:35.871567 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:35.871541    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"
Sep 04 23:00:36.078247 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.078198    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:36.121411 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.121385    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"]
Sep 04 23:00:36.122366 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:00:36.122340    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5431cfb5_73b3_4d0d_b5ff_062b7f7ff84e.slice/crio-46948fe67b71545ed771d8d94ef9e4050f923cbcd46c6edad61d67e74276a893 WatchSource:0}: Error finding container 46948fe67b71545ed771d8d94ef9e4050f923cbcd46c6edad61d67e74276a893: Status 404 returned error can't find the container with id 46948fe67b71545ed771d8d94ef9e4050f923cbcd46c6edad61d67e74276a893
Sep 04 23:00:36.166375 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.166158    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"]
Sep 04 23:00:36.304475 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.304440    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:36.405103 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.404968    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:36.505367 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.505279    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:36.763263 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.763087    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr" event={"ID":"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e","Type":"ContainerStarted","Data":"573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360"}
Sep 04 23:00:36.763263 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.763152    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr" event={"ID":"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e","Type":"ContainerStarted","Data":"46948fe67b71545ed771d8d94ef9e4050f923cbcd46c6edad61d67e74276a893"}
Sep 04 23:00:36.763263 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.763184    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"
Sep 04 23:00:36.765696 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.765587    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn" event={"ID":"a343ffe9-9686-449d-8e68-195643ea9565","Type":"ContainerStarted","Data":"abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da"}
Sep 04 23:00:36.765696 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.765660    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn" event={"ID":"a343ffe9-9686-449d-8e68-195643ea9565","Type":"ContainerStarted","Data":"9f0fc9643e1a8efbe0edace1e785a0e2a1bca8510ccea6d750574b5ed87f5596"}
Sep 04 23:00:36.765696 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.765693    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"
Sep 04 23:00:36.782143 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.782089    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr" podStartSLOduration=1.7820683929999999 podStartE2EDuration="1.782068393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:36.779578171 +0000 UTC m=+924.889698761" watchObservedRunningTime="2026-09-04 23:00:36.782068393 +0000 UTC m=+924.892188982"
Sep 04 23:00:36.801220 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:36.801170    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn" podStartSLOduration=1.801155506 podStartE2EDuration="1.801155506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:36.798960903 +0000 UTC m=+924.909081490" watchObservedRunningTime="2026-09-04 23:00:36.801155506 +0000 UTC m=+924.911276093"
Sep 04 23:00:37.080558 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:37.080525    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:38.425436 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:38.425399    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:00:38.425834 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:38.425481    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:00:38.425834 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:38.425529    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:00:38.783987 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:38.783900    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" event={"ID":"74d699b2-c57d-4386-88fe-794b2ff5fd90","Type":"ContainerStarted","Data":"c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1"}
Sep 04 23:00:38.805117 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:38.805064    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" podStartSLOduration=1.674315775 podStartE2EDuration="13.805047646s" totalImagesPullingTime="12.130731871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:38.803326007 +0000 UTC m=+926.913446594" watchObservedRunningTime="2026-09-04 23:00:38.805047646 +0000 UTC m=+926.915168231"
Sep 04 23:00:39.012359 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:39.012328    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:39.013801 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:39.013771    2578 patch_prober.go:28] interesting pod/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 04 23:00:39.013929 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:39.013824    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 04 23:00:39.139364 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:39.139319    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:40.015545 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:40.015514    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:40.216159 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:40.216125    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:40.319381 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:40.319296    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:40.748268 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:40.748231    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:40.803529 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:40.801170    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:40.808608 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:40.808572    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:41.190338 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:41.190301    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:41.288615 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:41.288578    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:41.551275 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:41.551188    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:41.657424 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:41.657385    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:41.963012 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:41.962979    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:42.186585 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.186549    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:42.281139 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.281052    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:42.304998 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.304969    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"]
Sep 04 23:00:42.308404 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.308387    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"
Sep 04 23:00:42.310868 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.310849    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-921d8a5d\""
Sep 04 23:00:42.318283 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.318261    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"]
Sep 04 23:00:42.367225 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.367191    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"]
Sep 04 23:00:42.371198 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.371176    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"
Sep 04 23:00:42.373588 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.373569    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-921d8a5d-dockercfg-st6sn\""
Sep 04 23:00:42.381393 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.381372    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"]
Sep 04 23:00:42.504858 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.504820    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76512ac9-2ed9-4f74-ada7-12d3743f951f-plugins-config-volume\") pod \"payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8\" (UID: \"76512ac9-2ed9-4f74-ada7-12d3743f951f\") " pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"
Sep 04 23:00:42.505033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.504903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j9c8\" (UniqueName: \"kubernetes.io/projected/c7b49d50-9768-459f-af1c-f0f4b2935578-kube-api-access-6j9c8\") pod \"payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46\" (UID: \"c7b49d50-9768-459f-af1c-f0f4b2935578\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"
Sep 04 23:00:42.505033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.504945    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b49d50-9768-459f-af1c-f0f4b2935578-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46\" (UID: \"c7b49d50-9768-459f-af1c-f0f4b2935578\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"
Sep 04 23:00:42.505124 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.505026    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnmr2\" (UniqueName: \"kubernetes.io/projected/76512ac9-2ed9-4f74-ada7-12d3743f951f-kube-api-access-tnmr2\") pod \"payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8\" (UID: \"76512ac9-2ed9-4f74-ada7-12d3743f951f\") " pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"
Sep 04 23:00:42.605521 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.605484    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnmr2\" (UniqueName: \"kubernetes.io/projected/76512ac9-2ed9-4f74-ada7-12d3743f951f-kube-api-access-tnmr2\") pod \"payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8\" (UID: \"76512ac9-2ed9-4f74-ada7-12d3743f951f\") " pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"
Sep 04 23:00:42.605704 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.605551    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76512ac9-2ed9-4f74-ada7-12d3743f951f-plugins-config-volume\") pod \"payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8\" (UID: \"76512ac9-2ed9-4f74-ada7-12d3743f951f\") " pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"
Sep 04 23:00:42.605704 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.605625    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j9c8\" (UniqueName: \"kubernetes.io/projected/c7b49d50-9768-459f-af1c-f0f4b2935578-kube-api-access-6j9c8\") pod \"payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46\" (UID: \"c7b49d50-9768-459f-af1c-f0f4b2935578\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"
Sep 04 23:00:42.605704 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.605689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b49d50-9768-459f-af1c-f0f4b2935578-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46\" (UID: \"c7b49d50-9768-459f-af1c-f0f4b2935578\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"
Sep 04 23:00:42.606479 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.606455    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76512ac9-2ed9-4f74-ada7-12d3743f951f-plugins-config-volume\") pod \"payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8\" (UID: \"76512ac9-2ed9-4f74-ada7-12d3743f951f\") " pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"
Sep 04 23:00:42.606592 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.606480    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b49d50-9768-459f-af1c-f0f4b2935578-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46\" (UID: \"c7b49d50-9768-459f-af1c-f0f4b2935578\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"
Sep 04 23:00:42.613413 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.613389    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnmr2\" (UniqueName: \"kubernetes.io/projected/76512ac9-2ed9-4f74-ada7-12d3743f951f-kube-api-access-tnmr2\") pod \"payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8\" (UID: \"76512ac9-2ed9-4f74-ada7-12d3743f951f\") " pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"
Sep 04 23:00:42.613518 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.613483    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j9c8\" (UniqueName: \"kubernetes.io/projected/c7b49d50-9768-459f-af1c-f0f4b2935578-kube-api-access-6j9c8\") pod \"payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46\" (UID: \"c7b49d50-9768-459f-af1c-f0f4b2935578\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"
Sep 04 23:00:42.620398 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.620379    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"
Sep 04 23:00:42.682954 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.682725    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"
Sep 04 23:00:42.895037 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.895008    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"]
Sep 04 23:00:42.895822 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:00:42.895800    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76512ac9_2ed9_4f74_ada7_12d3743f951f.slice/crio-c33a18a298b1e8ed0b484efbf68ac725912522967bf36d5eb451ccfb41dbeb0b WatchSource:0}: Error finding container c33a18a298b1e8ed0b484efbf68ac725912522967bf36d5eb451ccfb41dbeb0b: Status 404 returned error can't find the container with id c33a18a298b1e8ed0b484efbf68ac725912522967bf36d5eb451ccfb41dbeb0b
Sep 04 23:00:42.958362 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:42.958336    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"]
Sep 04 23:00:42.959203 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:00:42.959180    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b49d50_9768_459f_af1c_f0f4b2935578.slice/crio-fc5fe49695058eae2c388437c2ad6514cc8b6eeb2b70da63126f5c11584e316c WatchSource:0}: Error finding container fc5fe49695058eae2c388437c2ad6514cc8b6eeb2b70da63126f5c11584e316c: Status 404 returned error can't find the container with id fc5fe49695058eae2c388437c2ad6514cc8b6eeb2b70da63126f5c11584e316c
Sep 04 23:00:43.228874 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:43.228703    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:43.820475 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:43.820433    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46" event={"ID":"c7b49d50-9768-459f-af1c-f0f4b2935578","Type":"ContainerStarted","Data":"31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e"}
Sep 04 23:00:43.820973 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:43.820497    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46" event={"ID":"c7b49d50-9768-459f-af1c-f0f4b2935578","Type":"ContainerStarted","Data":"fc5fe49695058eae2c388437c2ad6514cc8b6eeb2b70da63126f5c11584e316c"}
Sep 04 23:00:43.820973 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:43.820626    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"
Sep 04 23:00:43.822134 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:43.822107    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8" event={"ID":"76512ac9-2ed9-4f74-ada7-12d3743f951f","Type":"ContainerStarted","Data":"c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1"}
Sep 04 23:00:43.822251 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:43.822153    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"
Sep 04 23:00:43.822251 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:43.822169    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8" event={"ID":"76512ac9-2ed9-4f74-ada7-12d3743f951f","Type":"ContainerStarted","Data":"c33a18a298b1e8ed0b484efbf68ac725912522967bf36d5eb451ccfb41dbeb0b"}
Sep 04 23:00:43.837655 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:43.837603    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46" podStartSLOduration=1.837591335 podStartE2EDuration="1.837591335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:43.835680509 +0000 UTC m=+931.945801123" watchObservedRunningTime="2026-09-04 23:00:43.837591335 +0000 UTC m=+931.947711923"
Sep 04 23:00:43.856771 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:43.856725    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8" podStartSLOduration=1.856705133 podStartE2EDuration="1.856705133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:43.855954212 +0000 UTC m=+931.966074805" watchObservedRunningTime="2026-09-04 23:00:43.856705133 +0000 UTC m=+931.966825719"
Sep 04 23:00:43.962955 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:43.962919    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:44.676576 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:44.676538    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:45.487331 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:45.487294    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:46.066757 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:46.066717    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:46.172080 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:46.172042    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:46.484546 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:46.484515    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:47.235104 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:47.235072    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:47.336123 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:47.336068    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:47.778495 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:47.778464    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"
Sep 04 23:00:47.779083 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:47.779041    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"
Sep 04 23:00:49.444960 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:49.444885    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:49.650683 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:49.650625    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:50.032263 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.032220    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:50.104982 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.104944    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:50.501405 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.501368    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:50.636668 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.634172    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"]
Sep 04 23:00:50.678556 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.678524    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"]
Sep 04 23:00:50.678756 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.678667    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"
Sep 04 23:00:50.681839 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.681708    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-4fd6f6\""
Sep 04 23:00:50.681839 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.681729    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-4fd6f6-dockercfg-5zxgp\""
Sep 04 23:00:50.786740 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.786659    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4f352f-5591-4f2d-bdf7-129518a5f9cc-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr\" (UID: \"4a4f352f-5591-4f2d-bdf7-129518a5f9cc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"
Sep 04 23:00:50.786740 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.786725    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwtjf\" (UniqueName: \"kubernetes.io/projected/4a4f352f-5591-4f2d-bdf7-129518a5f9cc-kube-api-access-pwtjf\") pod \"payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr\" (UID: \"4a4f352f-5591-4f2d-bdf7-129518a5f9cc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"
Sep 04 23:00:50.804120 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.804093    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:50.862433 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.862396    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:50.888033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.888002    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwtjf\" (UniqueName: \"kubernetes.io/projected/4a4f352f-5591-4f2d-bdf7-129518a5f9cc-kube-api-access-pwtjf\") pod \"payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr\" (UID: \"4a4f352f-5591-4f2d-bdf7-129518a5f9cc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"
Sep 04 23:00:50.888163 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.888098    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4f352f-5591-4f2d-bdf7-129518a5f9cc-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr\" (UID: \"4a4f352f-5591-4f2d-bdf7-129518a5f9cc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"
Sep 04 23:00:50.888832 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.888813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4f352f-5591-4f2d-bdf7-129518a5f9cc-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr\" (UID: \"4a4f352f-5591-4f2d-bdf7-129518a5f9cc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"
Sep 04 23:00:50.896480 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.896457    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwtjf\" (UniqueName: \"kubernetes.io/projected/4a4f352f-5591-4f2d-bdf7-129518a5f9cc-kube-api-access-pwtjf\") pod \"payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr\" (UID: \"4a4f352f-5591-4f2d-bdf7-129518a5f9cc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"
Sep 04 23:00:50.989481 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:50.989451    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"
Sep 04 23:00:51.289756 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:51.289726    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"]
Sep 04 23:00:51.291618 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:00:51.291588    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4f352f_5591_4f2d_bdf7_129518a5f9cc.slice/crio-17f84d955666de307b95930fb478580413dd7252865bbf1d8fee57c8a4f6823c WatchSource:0}: Error finding container 17f84d955666de307b95930fb478580413dd7252865bbf1d8fee57c8a4f6823c: Status 404 returned error can't find the container with id 17f84d955666de307b95930fb478580413dd7252865bbf1d8fee57c8a4f6823c
Sep 04 23:00:51.880393 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:51.880361    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr" event={"ID":"4a4f352f-5591-4f2d-bdf7-129518a5f9cc","Type":"ContainerStarted","Data":"399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f"}
Sep 04 23:00:51.880877 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:51.880411    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr" event={"ID":"4a4f352f-5591-4f2d-bdf7-129518a5f9cc","Type":"ContainerStarted","Data":"17f84d955666de307b95930fb478580413dd7252865bbf1d8fee57c8a4f6823c"}
Sep 04 23:00:51.881133 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:51.881113    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"
Sep 04 23:00:51.896853 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:51.896799    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr" podStartSLOduration=1.896784757 podStartE2EDuration="1.896784757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:51.895989331 +0000 UTC m=+940.006109918" watchObservedRunningTime="2026-09-04 23:00:51.896784757 +0000 UTC m=+940.006905344"
Sep 04 23:00:54.831115 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:54.831081    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"
Sep 04 23:00:54.831471 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:54.831178    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"
Sep 04 23:00:55.886309 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:55.886265    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:56.841867 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:56.841819    2578 patch_prober.go:28] interesting pod/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 04 23:00:56.842056 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:56.841894    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 04 23:00:57.920178 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:57.920135    2578 generic.go:356] "Generic (PLEG): container finished" podID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerID="c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1" exitCode=0
Sep 04 23:00:57.920739 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:57.920210    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" event={"ID":"74d699b2-c57d-4386-88fe-794b2ff5fd90","Type":"ContainerDied","Data":"c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1"}
Sep 04 23:00:57.920818 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:57.920777    2578 scope.go:117] "RemoveContainer" containerID="c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1"
Sep 04 23:00:57.921937 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:57.921906    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:00:57.922032 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:57.921976    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:00:57.922032 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:57.922006    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:00:58.647166 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:58.647128    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:00:58.927980 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:58.927891    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" event={"ID":"74d699b2-c57d-4386-88fe-794b2ff5fd90","Type":"ContainerStarted","Data":"f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed"}
Sep 04 23:00:59.012198 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:59.012158    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:00:59.012412 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:59.012384    2578 patch_prober.go:28] interesting pod/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 04 23:00:59.012498 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:00:59.012458    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 04 23:01:00.012317 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:00.012282    2578 patch_prober.go:28] interesting pod/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 04 23:01:00.012738 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:00.012340    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 04 23:01:01.012450 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:01.012415    2578 patch_prober.go:28] interesting pod/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 04 23:01:01.012934 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:01.012472    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 04 23:01:02.012046 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:02.012005    2578 patch_prober.go:28] interesting pod/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 04 23:01:02.012254 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:02.012072    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 04 23:01:02.639408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:02.639373    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:03.012278 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:03.012179    2578 patch_prober.go:28] interesting pod/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 04 23:01:03.012278 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:03.012245    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 04 23:01:03.894803 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:03.894763    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"
Sep 04 23:01:04.012582 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:04.012551    2578 patch_prober.go:28] interesting pod/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 04 23:01:04.012763 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:04.012605    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 04 23:01:05.015425 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:05.015394    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:01:05.015895 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:05.015757    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:01:05.016298 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:05.016277    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:01:10.882823 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:10.882791    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:11.104831 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:11.104795    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:13.797599 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:13.797561    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:21.747301 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:21.747260    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:22.462138 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.462068    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:22.493572 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.493540    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"]
Sep 04 23:01:22.493837 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.493810    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8" podUID="76512ac9-2ed9-4f74-ada7-12d3743f951f" containerName="payload-processing" containerID="cri-o://c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1" gracePeriod=30
Sep 04 23:01:22.500611 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.500580    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"]
Sep 04 23:01:22.501118 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.501017    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46" podUID="c7b49d50-9768-459f-af1c-f0f4b2935578" containerName="payload-pre-processing" containerID="cri-o://31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e" gracePeriod=30
Sep 04 23:01:22.532564 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.532538    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:22.587434 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.587402    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:22.671218 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.671177    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:22.746351 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.746306    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:22.767377 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.767354    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"
Sep 04 23:01:22.800397 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.800365    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"
Sep 04 23:01:22.802534 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.802508    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"]
Sep 04 23:01:22.802929 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.802900    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy" containerID="cri-o://f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed" gracePeriod=30
Sep 04 23:01:22.822178 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.822091    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:22.904750 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.904714    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6j9c8\" (UniqueName: \"kubernetes.io/projected/c7b49d50-9768-459f-af1c-f0f4b2935578-kube-api-access-6j9c8\") pod \"c7b49d50-9768-459f-af1c-f0f4b2935578\" (UID: \"c7b49d50-9768-459f-af1c-f0f4b2935578\") "
Sep 04 23:01:22.904940 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.904781    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b49d50-9768-459f-af1c-f0f4b2935578-plugins-config-volume\") pod \"c7b49d50-9768-459f-af1c-f0f4b2935578\" (UID: \"c7b49d50-9768-459f-af1c-f0f4b2935578\") "
Sep 04 23:01:22.904940 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.904863    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnmr2\" (UniqueName: \"kubernetes.io/projected/76512ac9-2ed9-4f74-ada7-12d3743f951f-kube-api-access-tnmr2\") pod \"76512ac9-2ed9-4f74-ada7-12d3743f951f\" (UID: \"76512ac9-2ed9-4f74-ada7-12d3743f951f\") "
Sep 04 23:01:22.904940 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.904884    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76512ac9-2ed9-4f74-ada7-12d3743f951f-plugins-config-volume\") pod \"76512ac9-2ed9-4f74-ada7-12d3743f951f\" (UID: \"76512ac9-2ed9-4f74-ada7-12d3743f951f\") "
Sep 04 23:01:22.905236 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.905201    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7b49d50-9768-459f-af1c-f0f4b2935578-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c7b49d50-9768-459f-af1c-f0f4b2935578" (UID: "c7b49d50-9768-459f-af1c-f0f4b2935578"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:01:22.905345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.905276    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76512ac9-2ed9-4f74-ada7-12d3743f951f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76512ac9-2ed9-4f74-ada7-12d3743f951f" (UID: "76512ac9-2ed9-4f74-ada7-12d3743f951f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:01:22.907195 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.907166    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7b49d50-9768-459f-af1c-f0f4b2935578-kube-api-access-6j9c8" (OuterVolumeSpecName: "kube-api-access-6j9c8") pod "c7b49d50-9768-459f-af1c-f0f4b2935578" (UID: "c7b49d50-9768-459f-af1c-f0f4b2935578"). InnerVolumeSpecName "kube-api-access-6j9c8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:01:22.907301 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:22.907213    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76512ac9-2ed9-4f74-ada7-12d3743f951f-kube-api-access-tnmr2" (OuterVolumeSpecName: "kube-api-access-tnmr2") pod "76512ac9-2ed9-4f74-ada7-12d3743f951f" (UID: "76512ac9-2ed9-4f74-ada7-12d3743f951f"). InnerVolumeSpecName "kube-api-access-tnmr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:01:23.006687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.006572    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnmr2\" (UniqueName: \"kubernetes.io/projected/76512ac9-2ed9-4f74-ada7-12d3743f951f-kube-api-access-tnmr2\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:23.006687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.006600    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76512ac9-2ed9-4f74-ada7-12d3743f951f-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:23.006687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.006609    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6j9c8\" (UniqueName: \"kubernetes.io/projected/c7b49d50-9768-459f-af1c-f0f4b2935578-kube-api-access-6j9c8\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:23.006687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.006617    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b49d50-9768-459f-af1c-f0f4b2935578-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:23.084834 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.084799    2578 generic.go:356] "Generic (PLEG): container finished" podID="c7b49d50-9768-459f-af1c-f0f4b2935578" containerID="31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e" exitCode=0
Sep 04 23:01:23.085009 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.084861    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"
Sep 04 23:01:23.085009 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.084890    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46" event={"ID":"c7b49d50-9768-459f-af1c-f0f4b2935578","Type":"ContainerDied","Data":"31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e"}
Sep 04 23:01:23.085009 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.084976    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46" event={"ID":"c7b49d50-9768-459f-af1c-f0f4b2935578","Type":"ContainerDied","Data":"fc5fe49695058eae2c388437c2ad6514cc8b6eeb2b70da63126f5c11584e316c"}
Sep 04 23:01:23.085147 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.085010    2578 scope.go:117] "RemoveContainer" containerID="31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e"
Sep 04 23:01:23.086373 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.086348    2578 generic.go:356] "Generic (PLEG): container finished" podID="76512ac9-2ed9-4f74-ada7-12d3743f951f" containerID="c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1" exitCode=0
Sep 04 23:01:23.086480 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.086388    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8" event={"ID":"76512ac9-2ed9-4f74-ada7-12d3743f951f","Type":"ContainerDied","Data":"c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1"}
Sep 04 23:01:23.086480 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.086411    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"
Sep 04 23:01:23.086480 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.086429    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8" event={"ID":"76512ac9-2ed9-4f74-ada7-12d3743f951f","Type":"ContainerDied","Data":"c33a18a298b1e8ed0b484efbf68ac725912522967bf36d5eb451ccfb41dbeb0b"}
Sep 04 23:01:23.095896 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.095880    2578 scope.go:117] "RemoveContainer" containerID="31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e"
Sep 04 23:01:23.096231 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:01:23.096201    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e\": container with ID starting with 31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e not found: ID does not exist" containerID="31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e"
Sep 04 23:01:23.096344 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.096242    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e"} err="failed to get container status \"31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e\": rpc error: code = NotFound desc = could not find container \"31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e\": container with ID starting with 31c931b730e68d6fe55ffa2495eb10dc85c3976c65048a968917246a101d6d3e not found: ID does not exist"
Sep 04 23:01:23.096344 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.096267    2578 scope.go:117] "RemoveContainer" containerID="c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1"
Sep 04 23:01:23.104906 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.104888    2578 scope.go:117] "RemoveContainer" containerID="c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1"
Sep 04 23:01:23.105164 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:01:23.105144    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1\": container with ID starting with c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1 not found: ID does not exist" containerID="c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1"
Sep 04 23:01:23.105203 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.105173    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1"} err="failed to get container status \"c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1\": rpc error: code = NotFound desc = could not find container \"c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1\": container with ID starting with c77082102a8d1082142db031f814326f05365de61b61e8e817a44a8b46dda9c1 not found: ID does not exist"
Sep 04 23:01:23.111356 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.111334    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"]
Sep 04 23:01:23.114885 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.114864    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-921d8a5d-7c6478769d-jnp46"]
Sep 04 23:01:23.125658 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.125612    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"]
Sep 04 23:01:23.128478 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.128458    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-921d8a5d-6cd67d7fdc-7h8f8"]
Sep 04 23:01:23.281794 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.281716    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:23.382275 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.382243    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:23.481418 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.481389    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:23.770240 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.770202    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"]
Sep 04 23:01:23.770886 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.770865    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76512ac9-2ed9-4f74-ada7-12d3743f951f" containerName="payload-processing"
Sep 04 23:01:23.770886 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.770889    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="76512ac9-2ed9-4f74-ada7-12d3743f951f" containerName="payload-processing"
Sep 04 23:01:23.771058 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.770928    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7b49d50-9768-459f-af1c-f0f4b2935578" containerName="payload-pre-processing"
Sep 04 23:01:23.771058 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.770935    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7b49d50-9768-459f-af1c-f0f4b2935578" containerName="payload-pre-processing"
Sep 04 23:01:23.771058 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.771027    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="76512ac9-2ed9-4f74-ada7-12d3743f951f" containerName="payload-processing"
Sep 04 23:01:23.771058 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.771036    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7b49d50-9768-459f-af1c-f0f4b2935578" containerName="payload-pre-processing"
Sep 04 23:01:23.775465 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.775437    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.779381 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.779357    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-4cc4c1dc-gw-openshift-default-dockercfg-frsnm\""
Sep 04 23:01:23.786134 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.786111    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"]
Sep 04 23:01:23.805818 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.805791    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:23.812779 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.812754    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-data\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.812904 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.812796    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istiod-ca-cert\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.812904 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.812861    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-token\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.813036 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.812987    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-workload-certs\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.813036 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.813017    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-workload-socket\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.813036 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.813033    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mlbj\" (UniqueName: \"kubernetes.io/projected/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-kube-api-access-4mlbj\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.813149 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.813053    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-credential-socket\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.813149 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.813068    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-envoy\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.813149 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.813086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-podinfo\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.914385 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.914334    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-credential-socket\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.914385 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.914387    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-envoy\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.914607 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.914472    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-podinfo\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.914607 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.914544    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-data\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.914607 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.914574    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istiod-ca-cert\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.914841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.914609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-token\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.914841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.914706    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-workload-certs\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.914841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.914736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-workload-socket\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.914841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.914764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mlbj\" (UniqueName: \"kubernetes.io/projected/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-kube-api-access-4mlbj\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.914841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.914782    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-credential-socket\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.915091 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.914945    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-data\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.915091 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.915057    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-workload-certs\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.915230 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.915202    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-workload-socket\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.915401 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.915382    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istiod-ca-cert\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.916889 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.916856    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-envoy\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.917100 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.917081    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-podinfo\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.922751 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.922729    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-token\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:23.922833 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:23.922764    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mlbj\" (UniqueName: \"kubernetes.io/projected/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-kube-api-access-4mlbj\") pod \"e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") " pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:24.091776 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:24.091743    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:24.235984 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:24.235948    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:24.373295 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:24.373262    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"]
Sep 04 23:01:24.374138 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:01:24.374112    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda7b4ecd_c8c4_4537_9a77_97bbe2d8ea4b.slice/crio-5e45bce689e67c6acf25d5d162d2930839d380951fffc979a124a0c53748f195 WatchSource:0}: Error finding container 5e45bce689e67c6acf25d5d162d2930839d380951fffc979a124a0c53748f195: Status 404 returned error can't find the container with id 5e45bce689e67c6acf25d5d162d2930839d380951fffc979a124a0c53748f195
Sep 04 23:01:24.376263 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:24.376232    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:01:24.376332 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:24.376298    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:01:24.376332 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:24.376327    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:01:24.421256 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:24.421227    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76512ac9-2ed9-4f74-ada7-12d3743f951f" path="/var/lib/kubelet/pods/76512ac9-2ed9-4f74-ada7-12d3743f951f/volumes"
Sep 04 23:01:24.421671 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:24.421632    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7b49d50-9768-459f-af1c-f0f4b2935578" path="/var/lib/kubelet/pods/c7b49d50-9768-459f-af1c-f0f4b2935578/volumes"
Sep 04 23:01:25.101359 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:25.101316    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" event={"ID":"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b","Type":"ContainerStarted","Data":"f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e"}
Sep 04 23:01:25.101780 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:25.101378    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" event={"ID":"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b","Type":"ContainerStarted","Data":"5e45bce689e67c6acf25d5d162d2930839d380951fffc979a124a0c53748f195"}
Sep 04 23:01:25.121842 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:25.121788    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" podStartSLOduration=2.12177348 podStartE2EDuration="2.12177348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:01:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:01:25.120503525 +0000 UTC m=+973.230624111" watchObservedRunningTime="2026-09-04 23:01:25.12177348 +0000 UTC m=+973.231894066"
Sep 04 23:01:26.092315 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:26.092280    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:26.097872 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:26.097847    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:26.107105 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:26.107087    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:26.107765 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:26.107746    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:26.564518 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:26.564433    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:26.671406 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:26.671354    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:26.765514 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:26.765478    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:27.091220 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:27.091187    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:27.223081 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:27.223050    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:27.321779 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:27.321748    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:28.074059 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.074038    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:01:28.122297 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.122265    2578 generic.go:356] "Generic (PLEG): container finished" podID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerID="f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed" exitCode=0
Sep 04 23:01:28.122440 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.122332    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"
Sep 04 23:01:28.122440 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.122339    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" event={"ID":"74d699b2-c57d-4386-88fe-794b2ff5fd90","Type":"ContainerDied","Data":"f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed"}
Sep 04 23:01:28.122440 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.122399    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft" event={"ID":"74d699b2-c57d-4386-88fe-794b2ff5fd90","Type":"ContainerDied","Data":"fb0d71009ea8762477473d675603fa71e00caf84fc14904b77e5f8e1bb66ed3a"}
Sep 04 23:01:28.122440 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.122429    2578 scope.go:117] "RemoveContainer" containerID="f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed"
Sep 04 23:01:28.131944 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.131927    2578 scope.go:117] "RemoveContainer" containerID="c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1"
Sep 04 23:01:28.143304 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.143282    2578 scope.go:117] "RemoveContainer" containerID="f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed"
Sep 04 23:01:28.143564 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:01:28.143545    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed\": container with ID starting with f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed not found: ID does not exist" containerID="f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed"
Sep 04 23:01:28.143614 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.143574    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed"} err="failed to get container status \"f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed\": rpc error: code = NotFound desc = could not find container \"f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed\": container with ID starting with f85cba6372af19fce7fea559cdac6ec6551b50f9e98b9f4d2cf1972a99f968ed not found: ID does not exist"
Sep 04 23:01:28.143614 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.143592    2578 scope.go:117] "RemoveContainer" containerID="c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1"
Sep 04 23:01:28.143875 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:01:28.143859    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1\": container with ID starting with c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1 not found: ID does not exist" containerID="c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1"
Sep 04 23:01:28.143928 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.143884    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1"} err="failed to get container status \"c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1\": rpc error: code = NotFound desc = could not find container \"c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1\": container with ID starting with c06d5f0e559051905f12f507c254591d9a4419dee3af6f5ebb85c1b393c480a1 not found: ID does not exist"
Sep 04 23:01:28.151283 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.151261    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-envoy\") pod \"74d699b2-c57d-4386-88fe-794b2ff5fd90\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") "
Sep 04 23:01:28.151382 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.151293    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfhlc\" (UniqueName: \"kubernetes.io/projected/74d699b2-c57d-4386-88fe-794b2ff5fd90-kube-api-access-pfhlc\") pod \"74d699b2-c57d-4386-88fe-794b2ff5fd90\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") "
Sep 04 23:01:28.151382 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.151312    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-workload-socket\") pod \"74d699b2-c57d-4386-88fe-794b2ff5fd90\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") "
Sep 04 23:01:28.151382 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.151341    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-podinfo\") pod \"74d699b2-c57d-4386-88fe-794b2ff5fd90\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") "
Sep 04 23:01:28.151543 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.151399    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74d699b2-c57d-4386-88fe-794b2ff5fd90-istiod-ca-cert\") pod \"74d699b2-c57d-4386-88fe-794b2ff5fd90\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") "
Sep 04 23:01:28.151543 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.151425    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-data\") pod \"74d699b2-c57d-4386-88fe-794b2ff5fd90\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") "
Sep 04 23:01:28.151543 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.151443    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-credential-socket\") pod \"74d699b2-c57d-4386-88fe-794b2ff5fd90\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") "
Sep 04 23:01:28.151543 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.151493    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-workload-certs\") pod \"74d699b2-c57d-4386-88fe-794b2ff5fd90\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") "
Sep 04 23:01:28.151543 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.151516    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-token\") pod \"74d699b2-c57d-4386-88fe-794b2ff5fd90\" (UID: \"74d699b2-c57d-4386-88fe-794b2ff5fd90\") "
Sep 04 23:01:28.151828 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.151614    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74d699b2-c57d-4386-88fe-794b2ff5fd90" (UID: "74d699b2-c57d-4386-88fe-794b2ff5fd90"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:01:28.151991 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.151969    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:28.152085 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.151999    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74d699b2-c57d-4386-88fe-794b2ff5fd90" (UID: "74d699b2-c57d-4386-88fe-794b2ff5fd90"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:01:28.152562 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.152365    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74d699b2-c57d-4386-88fe-794b2ff5fd90" (UID: "74d699b2-c57d-4386-88fe-794b2ff5fd90"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:01:28.152562 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.152528    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74d699b2-c57d-4386-88fe-794b2ff5fd90-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74d699b2-c57d-4386-88fe-794b2ff5fd90" (UID: "74d699b2-c57d-4386-88fe-794b2ff5fd90"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:01:28.152932 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.152826    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-data" (OuterVolumeSpecName: "istio-data") pod "74d699b2-c57d-4386-88fe-794b2ff5fd90" (UID: "74d699b2-c57d-4386-88fe-794b2ff5fd90"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:01:28.153865 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.153841    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74d699b2-c57d-4386-88fe-794b2ff5fd90" (UID: "74d699b2-c57d-4386-88fe-794b2ff5fd90"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:01:28.153969 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.153916    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-token" (OuterVolumeSpecName: "istio-token") pod "74d699b2-c57d-4386-88fe-794b2ff5fd90" (UID: "74d699b2-c57d-4386-88fe-794b2ff5fd90"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:01:28.154020 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.153972    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74d699b2-c57d-4386-88fe-794b2ff5fd90-kube-api-access-pfhlc" (OuterVolumeSpecName: "kube-api-access-pfhlc") pod "74d699b2-c57d-4386-88fe-794b2ff5fd90" (UID: "74d699b2-c57d-4386-88fe-794b2ff5fd90"). InnerVolumeSpecName "kube-api-access-pfhlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:01:28.154020 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.153995    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74d699b2-c57d-4386-88fe-794b2ff5fd90" (UID: "74d699b2-c57d-4386-88fe-794b2ff5fd90"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:01:28.253290 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.253258    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74d699b2-c57d-4386-88fe-794b2ff5fd90-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:28.253290 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.253287    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:28.253290 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.253295    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:28.253754 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.253305    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:28.253754 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.253313    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:28.253754 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.253321    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:28.253754 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.253329    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfhlc\" (UniqueName: \"kubernetes.io/projected/74d699b2-c57d-4386-88fe-794b2ff5fd90-kube-api-access-pfhlc\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:28.253754 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.253337    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74d699b2-c57d-4386-88fe-794b2ff5fd90-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:28.448459 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.448426    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"]
Sep 04 23:01:28.450962 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:28.450934    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-921d8a5d-openshift-default-d78cdcb64-vrrft"]
Sep 04 23:01:30.420015 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:30.419981    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" path="/var/lib/kubelet/pods/74d699b2-c57d-4386-88fe-794b2ff5fd90/volumes"
Sep 04 23:01:37.513971 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:37.513881    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:41.892133 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:41.892100    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:42.148856 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:42.148766    2578 patch_prober.go:28] interesting pod/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 04 23:01:42.148856 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:42.148837    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 04 23:01:43.219476 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:43.219443    2578 generic.go:356] "Generic (PLEG): container finished" podID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerID="f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e" exitCode=0
Sep 04 23:01:43.219912 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:43.219518    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" event={"ID":"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b","Type":"ContainerDied","Data":"f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e"}
Sep 04 23:01:43.219957 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:43.219932    2578 scope.go:117] "RemoveContainer" containerID="f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e"
Sep 04 23:01:43.221014 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:43.220977    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:01:43.221069 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:43.221060    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:01:43.221107 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:43.221091    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:01:44.227808 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:44.227769    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" event={"ID":"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b","Type":"ContainerStarted","Data":"da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df"}
Sep 04 23:01:45.092518 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.092474    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:45.092740 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.092714    2578 patch_prober.go:28] interesting pod/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 04 23:01:45.092803 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.092780    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 04 23:01:45.298033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.297994    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"]
Sep 04 23:01:45.298469 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.298454    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy"
Sep 04 23:01:45.298515 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.298471    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy"
Sep 04 23:01:45.298569 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.298559    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy"
Sep 04 23:01:45.298605 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.298570    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy"
Sep 04 23:01:45.301803 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.301788    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.305203 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.305180    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a64414\"/\"openshift-service-ca.crt\""
Sep 04 23:01:45.305320 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.305203    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a64414\"/\"default-dockercfg-9b8x9\""
Sep 04 23:01:45.305320 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.305180    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a64414\"/\"test-model-a64414-kserve-self-signed-certs\""
Sep 04 23:01:45.305516 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.305495    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a64414\"/\"kube-root-ca.crt\""
Sep 04 23:01:45.309953 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.309931    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"]
Sep 04 23:01:45.394419 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.394342    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:45.426289 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.426254    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjgzv\" (UniqueName: \"kubernetes.io/projected/79dd3773-f5a3-4063-873a-ccd5192f13ba-kube-api-access-tjgzv\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.426479 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.426305    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-model-cache\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.426479 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.426380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-dshm\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.426479 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.426443    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-home\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.426479 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.426468    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-tmp-dir\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.426720 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.426560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79dd3773-f5a3-4063-873a-ccd5192f13ba-tls-certs\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.527065 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.527028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-model-cache\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.527065 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.527069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-dshm\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.527323 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.527103    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-home\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.527323 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.527121    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-tmp-dir\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.527323 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.527155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79dd3773-f5a3-4063-873a-ccd5192f13ba-tls-certs\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.527323 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.527185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjgzv\" (UniqueName: \"kubernetes.io/projected/79dd3773-f5a3-4063-873a-ccd5192f13ba-kube-api-access-tjgzv\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.527531 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.527463    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-model-cache\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.527531 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.527486    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-home\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.527608 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.527527    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-tmp-dir\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.529388 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.529356    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-dshm\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.529627 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.529609    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79dd3773-f5a3-4063-873a-ccd5192f13ba-tls-certs\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.540442 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.540378    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjgzv\" (UniqueName: \"kubernetes.io/projected/79dd3773-f5a3-4063-873a-ccd5192f13ba-kube-api-access-tjgzv\") pod \"test-model-a64414-kserve-5b4b948f49-kn6fw\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") " pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.612555 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.612522    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:45.923174 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:45.923140    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"]
Sep 04 23:01:45.924754 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:01:45.924723    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79dd3773_f5a3_4063_873a_ccd5192f13ba.slice/crio-0072d332ea31f938de5e900f7b2f2849145c784f11c3231478197c1aa4396392 WatchSource:0}: Error finding container 0072d332ea31f938de5e900f7b2f2849145c784f11c3231478197c1aa4396392: Status 404 returned error can't find the container with id 0072d332ea31f938de5e900f7b2f2849145c784f11c3231478197c1aa4396392
Sep 04 23:01:46.092145 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:46.092103    2578 patch_prober.go:28] interesting pod/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 04 23:01:46.092298 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:46.092175    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 04 23:01:46.256880 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:46.256772    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw" event={"ID":"79dd3773-f5a3-4063-873a-ccd5192f13ba","Type":"ContainerStarted","Data":"edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f"}
Sep 04 23:01:46.256880 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:46.256831    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw" event={"ID":"79dd3773-f5a3-4063-873a-ccd5192f13ba","Type":"ContainerStarted","Data":"0072d332ea31f938de5e900f7b2f2849145c784f11c3231478197c1aa4396392"}
Sep 04 23:01:46.274262 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:46.274197    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw" podStartSLOduration=1.274176233 podStartE2EDuration="1.274176233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:01:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:01:46.272215857 +0000 UTC m=+994.382336446" watchObservedRunningTime="2026-09-04 23:01:46.274176233 +0000 UTC m=+994.384296820"
Sep 04 23:01:47.092757 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.092723    2578 patch_prober.go:28] interesting pod/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 04 23:01:47.093129 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.092785    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 04 23:01:47.596625 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.596594    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:47.700090 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.700054    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs"]
Sep 04 23:01:47.700503 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.700489    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy"
Sep 04 23:01:47.700550 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.700505    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="74d699b2-c57d-4386-88fe-794b2ff5fd90" containerName="istio-proxy"
Sep 04 23:01:47.703783 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.703760    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs"
Sep 04 23:01:47.706318 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.706287    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:01:47.712871 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.712848    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs"]
Sep 04 23:01:47.834631 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.834590    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:47.850613 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.850511    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54312668-838f-4ee3-8a8c-fb56d9314e83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-63d928-lbnxs\" (UID: \"54312668-838f-4ee3-8a8c-fb56d9314e83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs"
Sep 04 23:01:47.951363 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.951320    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54312668-838f-4ee3-8a8c-fb56d9314e83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-63d928-lbnxs\" (UID: \"54312668-838f-4ee3-8a8c-fb56d9314e83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs"
Sep 04 23:01:47.952231 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:47.952197    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54312668-838f-4ee3-8a8c-fb56d9314e83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-63d928-lbnxs\" (UID: \"54312668-838f-4ee3-8a8c-fb56d9314e83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs"
Sep 04 23:01:48.016793 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:48.016755    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs"
Sep 04 23:01:48.092918 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:48.092405    2578 patch_prober.go:28] interesting pod/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 04 23:01:48.092918 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:48.092474    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 04 23:01:48.315005 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:48.314979    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs"]
Sep 04 23:01:48.316690 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:01:48.316663    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54312668_838f_4ee3_8a8c_fb56d9314e83.slice/crio-c6639672e6b83218e35855c959c76a19e65141ce1c76e4f684c0323587ada670 WatchSource:0}: Error finding container c6639672e6b83218e35855c959c76a19e65141ce1c76e4f684c0323587ada670: Status 404 returned error can't find the container with id c6639672e6b83218e35855c959c76a19e65141ce1c76e4f684c0323587ada670
Sep 04 23:01:48.592027 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:48.591974    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:49.092308 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:49.092276    2578 patch_prober.go:28] interesting pod/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 04 23:01:49.092461 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:49.092331    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 04 23:01:49.169182 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:49.169148    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:49.277895 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:49.277862    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs" event={"ID":"54312668-838f-4ee3-8a8c-fb56d9314e83","Type":"ContainerStarted","Data":"91385f296bdb707f960c7180092e95f22e7905f4428ba959d9968e9dbbcaa1a1"}
Sep 04 23:01:49.278060 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:49.277913    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs" event={"ID":"54312668-838f-4ee3-8a8c-fb56d9314e83","Type":"ContainerStarted","Data":"c6639672e6b83218e35855c959c76a19e65141ce1c76e4f684c0323587ada670"}
Sep 04 23:01:49.293734 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:49.293681    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs" podStartSLOduration=2.29366086 podStartE2EDuration="2.29366086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:01:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:01:49.292048264 +0000 UTC m=+997.402168863" watchObservedRunningTime="2026-09-04 23:01:49.29366086 +0000 UTC m=+997.403781444"
Sep 04 23:01:50.154799 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:50.154762    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:50.155233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:50.155210    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:50.156152 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:50.156123    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:01:50.285210 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:50.285177    2578 generic.go:356] "Generic (PLEG): container finished" podID="54312668-838f-4ee3-8a8c-fb56d9314e83" containerID="91385f296bdb707f960c7180092e95f22e7905f4428ba959d9968e9dbbcaa1a1" exitCode=0
Sep 04 23:01:50.285612 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:50.285255    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs" event={"ID":"54312668-838f-4ee3-8a8c-fb56d9314e83","Type":"ContainerDied","Data":"91385f296bdb707f960c7180092e95f22e7905f4428ba959d9968e9dbbcaa1a1"}
Sep 04 23:01:51.435127 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.435100    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs"
Sep 04 23:01:51.580760 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.580725    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn"]
Sep 04 23:01:51.581172 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.581159    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54312668-838f-4ee3-8a8c-fb56d9314e83" containerName="revoke-keys"
Sep 04 23:01:51.581216 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.581173    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="54312668-838f-4ee3-8a8c-fb56d9314e83" containerName="revoke-keys"
Sep 04 23:01:51.581269 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.581260    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="54312668-838f-4ee3-8a8c-fb56d9314e83" containerName="revoke-keys"
Sep 04 23:01:51.584316 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.584299    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn"
Sep 04 23:01:51.595229 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.594534    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54312668-838f-4ee3-8a8c-fb56d9314e83-maas-api-service-ca\") pod \"54312668-838f-4ee3-8a8c-fb56d9314e83\" (UID: \"54312668-838f-4ee3-8a8c-fb56d9314e83\") "
Sep 04 23:01:51.595229 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.595126    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54312668-838f-4ee3-8a8c-fb56d9314e83-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "54312668-838f-4ee3-8a8c-fb56d9314e83" (UID: "54312668-838f-4ee3-8a8c-fb56d9314e83"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:01:51.595573 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.595546    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn"]
Sep 04 23:01:51.696201 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.696166    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed79037-6727-45cd-a554-2a9280502381-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn\" (UID: \"9ed79037-6727-45cd-a554-2a9280502381\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn"
Sep 04 23:01:51.696382 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.696221    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54312668-838f-4ee3-8a8c-fb56d9314e83-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:51.797524 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.797486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed79037-6727-45cd-a554-2a9280502381-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn\" (UID: \"9ed79037-6727-45cd-a554-2a9280502381\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn"
Sep 04 23:01:51.798139 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.798119    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed79037-6727-45cd-a554-2a9280502381-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn\" (UID: \"9ed79037-6727-45cd-a554-2a9280502381\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn"
Sep 04 23:01:51.896032 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:51.895947    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn"
Sep 04 23:01:52.170062 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:52.170037    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn"]
Sep 04 23:01:52.170683 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:01:52.170651    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed79037_6727_45cd_a554_2a9280502381.slice/crio-174f747c97b392dc5a4b06ccc418c5bdafe73a6358c58ca6c67c219cce2135ec WatchSource:0}: Error finding container 174f747c97b392dc5a4b06ccc418c5bdafe73a6358c58ca6c67c219cce2135ec: Status 404 returned error can't find the container with id 174f747c97b392dc5a4b06ccc418c5bdafe73a6358c58ca6c67c219cce2135ec
Sep 04 23:01:52.305710 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:52.305680    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs"
Sep 04 23:01:52.305893 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:52.305674    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs" event={"ID":"54312668-838f-4ee3-8a8c-fb56d9314e83","Type":"ContainerDied","Data":"c6639672e6b83218e35855c959c76a19e65141ce1c76e4f684c0323587ada670"}
Sep 04 23:01:52.305893 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:52.305835    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6639672e6b83218e35855c959c76a19e65141ce1c76e4f684c0323587ada670"
Sep 04 23:01:52.306932 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:52.306909    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn" event={"ID":"9ed79037-6727-45cd-a554-2a9280502381","Type":"ContainerStarted","Data":"174f747c97b392dc5a4b06ccc418c5bdafe73a6358c58ca6c67c219cce2135ec"}
Sep 04 23:01:53.314704 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:53.314602    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ed79037-6727-45cd-a554-2a9280502381" containerID="53fb450d0ab90244b26f92afbc0201c2ac126b871828447522f767572a1f9ca6" exitCode=0
Sep 04 23:01:53.314704 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:53.314691    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn" event={"ID":"9ed79037-6727-45cd-a554-2a9280502381","Type":"ContainerDied","Data":"53fb450d0ab90244b26f92afbc0201c2ac126b871828447522f767572a1f9ca6"}
Sep 04 23:01:54.465019 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:54.464993    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn"
Sep 04 23:01:54.627320 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:54.627286    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed79037-6727-45cd-a554-2a9280502381-maas-api-service-ca\") pod \"9ed79037-6727-45cd-a554-2a9280502381\" (UID: \"9ed79037-6727-45cd-a554-2a9280502381\") "
Sep 04 23:01:54.627737 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:54.627713    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ed79037-6727-45cd-a554-2a9280502381-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ed79037-6727-45cd-a554-2a9280502381" (UID: "9ed79037-6727-45cd-a554-2a9280502381"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:01:54.728485 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:54.728438    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed79037-6727-45cd-a554-2a9280502381-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:55.328087 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:55.328057    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn"
Sep 04 23:01:55.328289 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:55.328096    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn" event={"ID":"9ed79037-6727-45cd-a554-2a9280502381","Type":"ContainerDied","Data":"174f747c97b392dc5a4b06ccc418c5bdafe73a6358c58ca6c67c219cce2135ec"}
Sep 04 23:01:55.328289 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:55.328155    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="174f747c97b392dc5a4b06ccc418c5bdafe73a6358c58ca6c67c219cce2135ec"
Sep 04 23:01:55.612953 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:55.612856    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:55.612953 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:55.612935    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:55.626061 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:55.626034    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:56.345907 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:56.345879    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:01:57.397798 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:57.397769    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:57.497093 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:57.497064    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:57.597179 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:57.597136    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:57.604993 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:57.604963    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn"]
Sep 04 23:01:57.607494 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:57.607469    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc4c1dc-cczhn"]
Sep 04 23:01:57.697831 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:57.697728    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:57.739524 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:57.739487    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"]
Sep 04 23:01:57.739866 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:57.739836    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy" containerID="cri-o://da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df" gracePeriod=30
Sep 04 23:01:57.775565 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:57.775534    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:58.026007 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:58.025919    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:58.182701 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:58.182635    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:58.420585 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:58.420549    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ed79037-6727-45cd-a554-2a9280502381" path="/var/lib/kubelet/pods/9ed79037-6727-45cd-a554-2a9280502381/volumes"
Sep 04 23:01:58.513222 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:58.513190    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:58.883212 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:58.883178    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:58.947730 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:58.947699    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"]
Sep 04 23:01:58.947931 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:58.947905    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn" podUID="a343ffe9-9686-449d-8e68-195643ea9565" containerName="payload-processing" containerID="cri-o://abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da" gracePeriod=30
Sep 04 23:01:58.956608 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:58.955855    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"]
Sep 04 23:01:58.956608 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:58.956071    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr" podUID="5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e" containerName="payload-pre-processing" containerID="cri-o://573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360" gracePeriod=30
Sep 04 23:01:59.132011 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.131931    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs"]
Sep 04 23:01:59.136044 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.135923    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-63d928-lbnxs"]
Sep 04 23:01:59.212408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.212380    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"
Sep 04 23:01:59.254972 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.254938    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"
Sep 04 23:01:59.362607 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.362096    2578 generic.go:356] "Generic (PLEG): container finished" podID="a343ffe9-9686-449d-8e68-195643ea9565" containerID="abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da" exitCode=0
Sep 04 23:01:59.362607 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.362191    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn" event={"ID":"a343ffe9-9686-449d-8e68-195643ea9565","Type":"ContainerDied","Data":"abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da"}
Sep 04 23:01:59.362607 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.362242    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn" event={"ID":"a343ffe9-9686-449d-8e68-195643ea9565","Type":"ContainerDied","Data":"9f0fc9643e1a8efbe0edace1e785a0e2a1bca8510ccea6d750574b5ed87f5596"}
Sep 04 23:01:59.362607 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.362273    2578 scope.go:117] "RemoveContainer" containerID="abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da"
Sep 04 23:01:59.362607 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.362455    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"
Sep 04 23:01:59.364612 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.364589    2578 generic.go:356] "Generic (PLEG): container finished" podID="5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e" containerID="573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360" exitCode=0
Sep 04 23:01:59.364762 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.364626    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"
Sep 04 23:01:59.364762 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.364664    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr" event={"ID":"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e","Type":"ContainerDied","Data":"573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360"}
Sep 04 23:01:59.364762 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.364700    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr" event={"ID":"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e","Type":"ContainerDied","Data":"46948fe67b71545ed771d8d94ef9e4050f923cbcd46c6edad61d67e74276a893"}
Sep 04 23:01:59.373078 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.373052    2578 scope.go:117] "RemoveContainer" containerID="abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da"
Sep 04 23:01:59.373320 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.373297    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhjsg\" (UniqueName: \"kubernetes.io/projected/a343ffe9-9686-449d-8e68-195643ea9565-kube-api-access-fhjsg\") pod \"a343ffe9-9686-449d-8e68-195643ea9565\" (UID: \"a343ffe9-9686-449d-8e68-195643ea9565\") "
Sep 04 23:01:59.373398 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.373337    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pjhr\" (UniqueName: \"kubernetes.io/projected/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e-kube-api-access-5pjhr\") pod \"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e\" (UID: \"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e\") "
Sep 04 23:01:59.373398 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.373368    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a343ffe9-9686-449d-8e68-195643ea9565-plugins-config-volume\") pod \"a343ffe9-9686-449d-8e68-195643ea9565\" (UID: \"a343ffe9-9686-449d-8e68-195643ea9565\") "
Sep 04 23:01:59.373506 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.373406    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e-plugins-config-volume\") pod \"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e\" (UID: \"5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e\") "
Sep 04 23:01:59.373567 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:01:59.373538    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da\": container with ID starting with abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da not found: ID does not exist" containerID="abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da"
Sep 04 23:01:59.373807 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.373602    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da"} err="failed to get container status \"abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da\": rpc error: code = NotFound desc = could not find container \"abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da\": container with ID starting with abf9fca16a5dd33fac6ec428f99b3bd7099daf42bf446d58e7e01182eee792da not found: ID does not exist"
Sep 04 23:01:59.373807 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.373708    2578 scope.go:117] "RemoveContainer" containerID="573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360"
Sep 04 23:01:59.374241 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.374170    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e" (UID: "5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:01:59.374303 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.374267    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a343ffe9-9686-449d-8e68-195643ea9565-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a343ffe9-9686-449d-8e68-195643ea9565" (UID: "a343ffe9-9686-449d-8e68-195643ea9565"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:01:59.377804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.375806    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e-kube-api-access-5pjhr" (OuterVolumeSpecName: "kube-api-access-5pjhr") pod "5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e" (UID: "5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e"). InnerVolumeSpecName "kube-api-access-5pjhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:01:59.377804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.376270    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a343ffe9-9686-449d-8e68-195643ea9565-kube-api-access-fhjsg" (OuterVolumeSpecName: "kube-api-access-fhjsg") pod "a343ffe9-9686-449d-8e68-195643ea9565" (UID: "a343ffe9-9686-449d-8e68-195643ea9565"). InnerVolumeSpecName "kube-api-access-fhjsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:01:59.386916 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.386899    2578 scope.go:117] "RemoveContainer" containerID="573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360"
Sep 04 23:01:59.387208 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:01:59.387188    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360\": container with ID starting with 573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360 not found: ID does not exist" containerID="573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360"
Sep 04 23:01:59.387262 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.387215    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360"} err="failed to get container status \"573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360\": rpc error: code = NotFound desc = could not find container \"573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360\": container with ID starting with 573739c0e63a02cd7993a4346527b4273619cda0cee78cc993b17237b0428360 not found: ID does not exist"
Sep 04 23:01:59.388616 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.388592    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:59.448150 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.448112    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:59.480883 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.475020    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:59.480883 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.475053    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhjsg\" (UniqueName: \"kubernetes.io/projected/a343ffe9-9686-449d-8e68-195643ea9565-kube-api-access-fhjsg\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:59.480883 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.475087    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pjhr\" (UniqueName: \"kubernetes.io/projected/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e-kube-api-access-5pjhr\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:59.480883 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.475109    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a343ffe9-9686-449d-8e68-195643ea9565-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:01:59.512866 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.512832    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:01:59.685880 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.685852    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"]
Sep 04 23:01:59.690793 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.690768    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-63d928-6d65b6448f-9rvnn"]
Sep 04 23:01:59.700995 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.700973    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"]
Sep 04 23:01:59.704786 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:01:59.704761    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-63d928-5cd984d74-g5qvr"]
Sep 04 23:02:00.420005 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:00.419975    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54312668-838f-4ee3-8a8c-fb56d9314e83" path="/var/lib/kubelet/pods/54312668-838f-4ee3-8a8c-fb56d9314e83/volumes"
Sep 04 23:02:00.420360 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:00.420346    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e" path="/var/lib/kubelet/pods/5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e/volumes"
Sep 04 23:02:00.420716 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:00.420701    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a343ffe9-9686-449d-8e68-195643ea9565" path="/var/lib/kubelet/pods/a343ffe9-9686-449d-8e68-195643ea9565/volumes"
Sep 04 23:02:00.683114 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:00.683023    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:00.781558 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:00.781524    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:01.834337 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:01.834306    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:01.935234 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:01.935204    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:02.260150 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.260056    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:02.442108 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.442081    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:02.560343 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.560262    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:02.761362 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.761328    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"]
Sep 04 23:02:02.761805 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.761788    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a343ffe9-9686-449d-8e68-195643ea9565" containerName="payload-processing"
Sep 04 23:02:02.761805 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.761806    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a343ffe9-9686-449d-8e68-195643ea9565" containerName="payload-processing"
Sep 04 23:02:02.761937 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.761824    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed79037-6727-45cd-a554-2a9280502381" containerName="revoke-keys"
Sep 04 23:02:02.761937 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.761829    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed79037-6727-45cd-a554-2a9280502381" containerName="revoke-keys"
Sep 04 23:02:02.761937 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.761846    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e" containerName="payload-pre-processing"
Sep 04 23:02:02.761937 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.761852    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e" containerName="payload-pre-processing"
Sep 04 23:02:02.761937 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.761912    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5431cfb5-73b3-4d0d-b5ff-062b7f7ff84e" containerName="payload-pre-processing"
Sep 04 23:02:02.761937 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.761920    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a343ffe9-9686-449d-8e68-195643ea9565" containerName="payload-processing"
Sep 04 23:02:02.761937 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.761927    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ed79037-6727-45cd-a554-2a9280502381" containerName="revoke-keys"
Sep 04 23:02:02.765171 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.765143    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"
Sep 04 23:02:02.767896 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.767871    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e8480dce-dockercfg-nrthq\""
Sep 04 23:02:02.768004 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.767871    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e8480dce\""
Sep 04 23:02:02.776963 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.776942    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"]
Sep 04 23:02:02.806627 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.806589    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lswkp\" (UniqueName: \"kubernetes.io/projected/2fff9f17-de3f-46f0-801a-032b10d25e06-kube-api-access-lswkp\") pod \"payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7\" (UID: \"2fff9f17-de3f-46f0-801a-032b10d25e06\") " pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"
Sep 04 23:02:02.806802 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.806684    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fff9f17-de3f-46f0-801a-032b10d25e06-plugins-config-volume\") pod \"payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7\" (UID: \"2fff9f17-de3f-46f0-801a-032b10d25e06\") " pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"
Sep 04 23:02:02.907760 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.907725    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lswkp\" (UniqueName: \"kubernetes.io/projected/2fff9f17-de3f-46f0-801a-032b10d25e06-kube-api-access-lswkp\") pod \"payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7\" (UID: \"2fff9f17-de3f-46f0-801a-032b10d25e06\") " pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"
Sep 04 23:02:02.907760 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.907762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fff9f17-de3f-46f0-801a-032b10d25e06-plugins-config-volume\") pod \"payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7\" (UID: \"2fff9f17-de3f-46f0-801a-032b10d25e06\") " pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"
Sep 04 23:02:02.908725 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.908705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fff9f17-de3f-46f0-801a-032b10d25e06-plugins-config-volume\") pod \"payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7\" (UID: \"2fff9f17-de3f-46f0-801a-032b10d25e06\") " pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"
Sep 04 23:02:02.916635 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:02.916598    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lswkp\" (UniqueName: \"kubernetes.io/projected/2fff9f17-de3f-46f0-801a-032b10d25e06-kube-api-access-lswkp\") pod \"payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7\" (UID: \"2fff9f17-de3f-46f0-801a-032b10d25e06\") " pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"
Sep 04 23:02:03.012506 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.012479    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:02:03.109408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.109318    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mlbj\" (UniqueName: \"kubernetes.io/projected/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-kube-api-access-4mlbj\") pod \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") "
Sep 04 23:02:03.109408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.109359    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-credential-socket\") pod \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") "
Sep 04 23:02:03.109601 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.109419    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-token\") pod \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") "
Sep 04 23:02:03.109601 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.109586    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-data\") pod \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") "
Sep 04 23:02:03.109706 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.109628    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-workload-socket\") pod \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") "
Sep 04 23:02:03.109706 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.109687    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-podinfo\") pod \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") "
Sep 04 23:02:03.109802 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.109723    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" (UID: "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:02:03.109802 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.109729    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-envoy\") pod \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") "
Sep 04 23:02:03.109802 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.109793    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-workload-certs\") pod \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") "
Sep 04 23:02:03.109926 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.109869    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istiod-ca-cert\") pod \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\" (UID: \"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b\") "
Sep 04 23:02:03.109926 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.109879    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-data" (OuterVolumeSpecName: "istio-data") pod "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" (UID: "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:02:03.110130 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.109986    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" (UID: "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:02:03.110130 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.110072    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" (UID: "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:02:03.110296 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.110251    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:02:03.110296 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.110273    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:02:03.110296 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.110289    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:02:03.110407 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.110304    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:02:03.110407 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.110301    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" (UID: "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:02:03.111629 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.111606    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-kube-api-access-4mlbj" (OuterVolumeSpecName: "kube-api-access-4mlbj") pod "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" (UID: "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b"). InnerVolumeSpecName "kube-api-access-4mlbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:02:03.111904 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.111871    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" (UID: "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:02:03.112084 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.112042    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-token" (OuterVolumeSpecName: "istio-token") pod "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" (UID: "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:02:03.112791 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.112763    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" (UID: "da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:02:03.134493 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.134459    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"
Sep 04 23:02:03.211111 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.211069    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mlbj\" (UniqueName: \"kubernetes.io/projected/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-kube-api-access-4mlbj\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:02:03.211111 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.211107    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:02:03.211380 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.211128    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:02:03.211380 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.211142    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:02:03.211380 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.211156    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:02:03.394512 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.394422    2578 generic.go:356] "Generic (PLEG): container finished" podID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerID="da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df" exitCode=0
Sep 04 23:02:03.394512 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.394477    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"
Sep 04 23:02:03.394732 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.394507    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" event={"ID":"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b","Type":"ContainerDied","Data":"da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df"}
Sep 04 23:02:03.394732 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.394563    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l" event={"ID":"da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b","Type":"ContainerDied","Data":"5e45bce689e67c6acf25d5d162d2930839d380951fffc979a124a0c53748f195"}
Sep 04 23:02:03.394732 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.394586    2578 scope.go:117] "RemoveContainer" containerID="da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df"
Sep 04 23:02:03.406393 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.406365    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"]
Sep 04 23:02:03.406577 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.406562    2578 scope.go:117] "RemoveContainer" containerID="f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e"
Sep 04 23:02:03.407541 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:02:03.407517    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fff9f17_de3f_46f0_801a_032b10d25e06.slice/crio-c6bbeb590555d5246191a53a0c964316fc802a7d1f9605961975f3bcbf5986fc WatchSource:0}: Error finding container c6bbeb590555d5246191a53a0c964316fc802a7d1f9605961975f3bcbf5986fc: Status 404 returned error can't find the container with id c6bbeb590555d5246191a53a0c964316fc802a7d1f9605961975f3bcbf5986fc
Sep 04 23:02:03.416228 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.416206    2578 scope.go:117] "RemoveContainer" containerID="da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df"
Sep 04 23:02:03.416577 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:02:03.416557    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df\": container with ID starting with da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df not found: ID does not exist" containerID="da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df"
Sep 04 23:02:03.416725 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.416591    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df"} err="failed to get container status \"da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df\": rpc error: code = NotFound desc = could not find container \"da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df\": container with ID starting with da248e82383aeadf3759c06fa78923707587c6da973f2503b862e5a7db4c14df not found: ID does not exist"
Sep 04 23:02:03.416725 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.416616    2578 scope.go:117] "RemoveContainer" containerID="f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e"
Sep 04 23:02:03.416982 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:02:03.416964    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e\": container with ID starting with f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e not found: ID does not exist" containerID="f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e"
Sep 04 23:02:03.417047 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.416991    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e"} err="failed to get container status \"f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e\": rpc error: code = NotFound desc = could not find container \"f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e\": container with ID starting with f581d29e1983dcc00fc62b38947226c569fcbc7443da79fdbd52823b31a38a1e not found: ID does not exist"
Sep 04 23:02:03.417898 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.417879    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"]
Sep 04 23:02:03.423438 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:03.423406    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-4cc4c1dc-gw-openshift-default-645fbf4bb8-lwp8l"]
Sep 04 23:02:04.404519 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:04.404415    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7" event={"ID":"2fff9f17-de3f-46f0-801a-032b10d25e06","Type":"ContainerStarted","Data":"3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118"}
Sep 04 23:02:04.404519 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:04.404480    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7" event={"ID":"2fff9f17-de3f-46f0-801a-032b10d25e06","Type":"ContainerStarted","Data":"c6bbeb590555d5246191a53a0c964316fc802a7d1f9605961975f3bcbf5986fc"}
Sep 04 23:02:04.405030 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:04.404589    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"
Sep 04 23:02:04.419912 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:04.419879    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" path="/var/lib/kubelet/pods/da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b/volumes"
Sep 04 23:02:04.423657 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:04.423594    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7" podStartSLOduration=2.42357882 podStartE2EDuration="2.42357882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:02:04.422693516 +0000 UTC m=+1012.532814127" watchObservedRunningTime="2026-09-04 23:02:04.42357882 +0000 UTC m=+1012.533699406"
Sep 04 23:02:05.443480 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:05.443439    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:05.510824 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:05.510785    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:05.713966 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:05.713866    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:06.180250 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.180209    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:06.474319 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.474195    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"]
Sep 04 23:02:06.474694 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.474674    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy"
Sep 04 23:02:06.474694 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.474689    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy"
Sep 04 23:02:06.474792 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.474784    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy"
Sep 04 23:02:06.474836 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.474797    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy"
Sep 04 23:02:06.479116 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.479098    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.481617 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.481598    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-72563d-openshift-default-dockercfg-s7bwj\""
Sep 04 23:02:06.489487 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.489464    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"]
Sep 04 23:02:06.503911 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.503889    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:06.540116 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.540083    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6974e567-2f6c-468a-9a35-64fc79e91e61-istiod-ca-cert\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.540267 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.540131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-credential-socket\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.540267 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.540162    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-podinfo\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.540267 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.540218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-workload-socket\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.540267 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.540236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-envoy\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.540409 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.540306    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-data\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.540409 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.540374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-workload-certs\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.540409 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.540400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-token\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.540507 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.540437    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lllm\" (UniqueName: \"kubernetes.io/projected/6974e567-2f6c-468a-9a35-64fc79e91e61-kube-api-access-6lllm\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.641235 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.641195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6974e567-2f6c-468a-9a35-64fc79e91e61-istiod-ca-cert\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.641390 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.641273    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-credential-socket\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.641390 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.641297    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-podinfo\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.641390 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.641334    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-workload-socket\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.641390 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.641357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-envoy\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.641390 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.641386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-data\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.641724 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.641417    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-workload-certs\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.641724 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.641462    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-token\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.641724 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.641528    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lllm\" (UniqueName: \"kubernetes.io/projected/6974e567-2f6c-468a-9a35-64fc79e91e61-kube-api-access-6lllm\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.641893 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.641815    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-credential-socket\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.641893 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.641864    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-workload-socket\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.642095 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.642061    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-data\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.642251 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.642231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6974e567-2f6c-468a-9a35-64fc79e91e61-istiod-ca-cert\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.652041 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.642209    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-workload-certs\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.652041 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.644257    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-envoy\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.652041 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.648424    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-podinfo\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.652041 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.651622    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-token\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.655983 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.655959    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lllm\" (UniqueName: \"kubernetes.io/projected/6974e567-2f6c-468a-9a35-64fc79e91e61-kube-api-access-6lllm\") pod \"e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") " pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:06.780371 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.780287    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:06.791293 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:06.791265    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:07.071612 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:07.071580    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"]
Sep 04 23:02:07.073265 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:02:07.073237    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6974e567_2f6c_468a_9a35_64fc79e91e61.slice/crio-c950a1cf8dee9a91db6fa80e695213b7fc7c621f8f8372d8a27161998eb84b8f WatchSource:0}: Error finding container c950a1cf8dee9a91db6fa80e695213b7fc7c621f8f8372d8a27161998eb84b8f: Status 404 returned error can't find the container with id c950a1cf8dee9a91db6fa80e695213b7fc7c621f8f8372d8a27161998eb84b8f
Sep 04 23:02:07.075570 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:07.075534    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:02:07.075674 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:07.075602    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:02:07.075674 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:07.075633    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:02:07.427169 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:07.427135    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" event={"ID":"6974e567-2f6c-468a-9a35-64fc79e91e61","Type":"ContainerStarted","Data":"3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a"}
Sep 04 23:02:07.427462 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:07.427182    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" event={"ID":"6974e567-2f6c-468a-9a35-64fc79e91e61","Type":"ContainerStarted","Data":"c950a1cf8dee9a91db6fa80e695213b7fc7c621f8f8372d8a27161998eb84b8f"}
Sep 04 23:02:07.446774 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:07.446723    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" podStartSLOduration=1.446707474 podStartE2EDuration="1.446707474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:02:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:02:07.445079783 +0000 UTC m=+1015.555200369" watchObservedRunningTime="2026-09-04 23:02:07.446707474 +0000 UTC m=+1015.556828060"
Sep 04 23:02:07.792394 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:07.792308    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:08.796876 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:08.796843    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:09.357536 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:09.357499    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:09.439775 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:09.439746    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:09.440699 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:09.440672    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:09.463122 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:09.463088    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:10.522561 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:10.522530    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:10.582311 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:10.582283    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:10.683354 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:10.683325    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:11.031432 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:11.031402    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:11.225565 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:11.225537    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:15.414421 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:15.414389    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"
Sep 04 23:02:23.018406 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:23.018371    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:26.322159 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:26.322121    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q"]
Sep 04 23:02:26.322605 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:26.322542    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy"
Sep 04 23:02:26.322605 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:26.322553    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="da7b4ecd-c8c4-4537-9a77-97bbe2d8ea4b" containerName="istio-proxy"
Sep 04 23:02:26.327631 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:26.327611    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q"
Sep 04 23:02:26.331482 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:26.330843    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:02:26.334099 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:26.334033    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q"]
Sep 04 23:02:26.435090 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:26.435054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/353120ad-a9d1-4142-afe1-d64f8d6a6ea6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q\" (UID: \"353120ad-a9d1-4142-afe1-d64f8d6a6ea6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q"
Sep 04 23:02:26.536092 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:26.536049    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/353120ad-a9d1-4142-afe1-d64f8d6a6ea6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q\" (UID: \"353120ad-a9d1-4142-afe1-d64f8d6a6ea6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q"
Sep 04 23:02:26.536962 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:26.536938    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/353120ad-a9d1-4142-afe1-d64f8d6a6ea6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q\" (UID: \"353120ad-a9d1-4142-afe1-d64f8d6a6ea6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q"
Sep 04 23:02:26.641628 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:26.641591    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q"
Sep 04 23:02:26.919319 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:26.919282    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q"]
Sep 04 23:02:26.920795 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:02:26.920769    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353120ad_a9d1_4142_afe1_d64f8d6a6ea6.slice/crio-2656903959cd11a906d9d295fa65e71a2901294c32bc1b971edff9b047afb6e1 WatchSource:0}: Error finding container 2656903959cd11a906d9d295fa65e71a2901294c32bc1b971edff9b047afb6e1: Status 404 returned error can't find the container with id 2656903959cd11a906d9d295fa65e71a2901294c32bc1b971edff9b047afb6e1
Sep 04 23:02:27.556308 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:27.556215    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q" event={"ID":"353120ad-a9d1-4142-afe1-d64f8d6a6ea6","Type":"ContainerStarted","Data":"214cdf2da09983295415e11847220a2e46e91fd0ef77cc0d8c4cd024adfd0ada"}
Sep 04 23:02:27.556308 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:27.556287    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q" event={"ID":"353120ad-a9d1-4142-afe1-d64f8d6a6ea6","Type":"ContainerStarted","Data":"2656903959cd11a906d9d295fa65e71a2901294c32bc1b971edff9b047afb6e1"}
Sep 04 23:02:27.572121 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:27.572050    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q" podStartSLOduration=1.572036242 podStartE2EDuration="1.572036242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:02:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:02:27.571482681 +0000 UTC m=+1035.681603270" watchObservedRunningTime="2026-09-04 23:02:27.572036242 +0000 UTC m=+1035.682156828"
Sep 04 23:02:28.563522 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:28.563421    2578 generic.go:356] "Generic (PLEG): container finished" podID="353120ad-a9d1-4142-afe1-d64f8d6a6ea6" containerID="214cdf2da09983295415e11847220a2e46e91fd0ef77cc0d8c4cd024adfd0ada" exitCode=0
Sep 04 23:02:28.563980 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:28.563517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q" event={"ID":"353120ad-a9d1-4142-afe1-d64f8d6a6ea6","Type":"ContainerDied","Data":"214cdf2da09983295415e11847220a2e46e91fd0ef77cc0d8c4cd024adfd0ada"}
Sep 04 23:02:28.565166 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:28.565146    2578 generic.go:356] "Generic (PLEG): container finished" podID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerID="3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a" exitCode=0
Sep 04 23:02:28.565272 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:28.565181    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" event={"ID":"6974e567-2f6c-468a-9a35-64fc79e91e61","Type":"ContainerDied","Data":"3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a"}
Sep 04 23:02:28.565526 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:28.565513    2578 scope.go:117] "RemoveContainer" containerID="3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a"
Sep 04 23:02:28.566555 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:28.566526    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:02:28.566663 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:28.566625    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:02:28.566743 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:28.566695    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:02:29.574800 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:29.574758    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" event={"ID":"6974e567-2f6c-468a-9a35-64fc79e91e61","Type":"ContainerStarted","Data":"2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd"}
Sep 04 23:02:29.725566 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:29.725538    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q"
Sep 04 23:02:29.792159 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:29.792116    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:29.792363 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:29.792335    2578 patch_prober.go:28] interesting pod/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 04 23:02:29.792450 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:29.792408    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 04 23:02:29.869035 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:29.868939    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/353120ad-a9d1-4142-afe1-d64f8d6a6ea6-maas-api-service-ca\") pod \"353120ad-a9d1-4142-afe1-d64f8d6a6ea6\" (UID: \"353120ad-a9d1-4142-afe1-d64f8d6a6ea6\") "
Sep 04 23:02:29.869315 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:29.869287    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353120ad-a9d1-4142-afe1-d64f8d6a6ea6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "353120ad-a9d1-4142-afe1-d64f8d6a6ea6" (UID: "353120ad-a9d1-4142-afe1-d64f8d6a6ea6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:02:29.970572 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:29.970533    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/353120ad-a9d1-4142-afe1-d64f8d6a6ea6-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:02:30.582464 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:30.582433    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q"
Sep 04 23:02:30.582895 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:30.582473    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q" event={"ID":"353120ad-a9d1-4142-afe1-d64f8d6a6ea6","Type":"ContainerDied","Data":"2656903959cd11a906d9d295fa65e71a2901294c32bc1b971edff9b047afb6e1"}
Sep 04 23:02:30.582895 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:30.582540    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2656903959cd11a906d9d295fa65e71a2901294c32bc1b971edff9b047afb6e1"
Sep 04 23:02:30.792228 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:30.792136    2578 patch_prober.go:28] interesting pod/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 04 23:02:30.792228 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:30.792210    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 04 23:02:31.530466 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:31.530432    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:31.792220 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:31.792126    2578 patch_prober.go:28] interesting pod/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 04 23:02:31.792220 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:31.792196    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 04 23:02:32.187792 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.187752    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"]
Sep 04 23:02:32.188096 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.188030    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7" podUID="2fff9f17-de3f-46f0-801a-032b10d25e06" containerName="payload-processing" containerID="cri-o://3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118" gracePeriod=30
Sep 04 23:02:32.381387 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.381343    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q"]
Sep 04 23:02:32.383840 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.383812    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8480dce-9wl9q"]
Sep 04 23:02:32.422695 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.422662    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="353120ad-a9d1-4142-afe1-d64f8d6a6ea6" path="/var/lib/kubelet/pods/353120ad-a9d1-4142-afe1-d64f8d6a6ea6/volumes"
Sep 04 23:02:32.488850 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.488825    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"
Sep 04 23:02:32.565477 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.565441    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:32.596174 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.596133    2578 generic.go:356] "Generic (PLEG): container finished" podID="2fff9f17-de3f-46f0-801a-032b10d25e06" containerID="3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118" exitCode=0
Sep 04 23:02:32.596334 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.596183    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"
Sep 04 23:02:32.596334 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.596208    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7" event={"ID":"2fff9f17-de3f-46f0-801a-032b10d25e06","Type":"ContainerDied","Data":"3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118"}
Sep 04 23:02:32.596334 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.596256    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7" event={"ID":"2fff9f17-de3f-46f0-801a-032b10d25e06","Type":"ContainerDied","Data":"c6bbeb590555d5246191a53a0c964316fc802a7d1f9605961975f3bcbf5986fc"}
Sep 04 23:02:32.596334 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.596281    2578 scope.go:117] "RemoveContainer" containerID="3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118"
Sep 04 23:02:32.600920 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.600898    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lswkp\" (UniqueName: \"kubernetes.io/projected/2fff9f17-de3f-46f0-801a-032b10d25e06-kube-api-access-lswkp\") pod \"2fff9f17-de3f-46f0-801a-032b10d25e06\" (UID: \"2fff9f17-de3f-46f0-801a-032b10d25e06\") "
Sep 04 23:02:32.601038 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.600949    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fff9f17-de3f-46f0-801a-032b10d25e06-plugins-config-volume\") pod \"2fff9f17-de3f-46f0-801a-032b10d25e06\" (UID: \"2fff9f17-de3f-46f0-801a-032b10d25e06\") "
Sep 04 23:02:32.601347 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.601321    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fff9f17-de3f-46f0-801a-032b10d25e06-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2fff9f17-de3f-46f0-801a-032b10d25e06" (UID: "2fff9f17-de3f-46f0-801a-032b10d25e06"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:02:32.603042 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.603021    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fff9f17-de3f-46f0-801a-032b10d25e06-kube-api-access-lswkp" (OuterVolumeSpecName: "kube-api-access-lswkp") pod "2fff9f17-de3f-46f0-801a-032b10d25e06" (UID: "2fff9f17-de3f-46f0-801a-032b10d25e06"). InnerVolumeSpecName "kube-api-access-lswkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:02:32.605712 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.605690    2578 scope.go:117] "RemoveContainer" containerID="3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118"
Sep 04 23:02:32.605990 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:02:32.605973    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118\": container with ID starting with 3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118 not found: ID does not exist" containerID="3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118"
Sep 04 23:02:32.606040 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.605998    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118"} err="failed to get container status \"3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118\": rpc error: code = NotFound desc = could not find container \"3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118\": container with ID starting with 3f6c5b953bf97f816da1805bc4f49f1cca223c67c0516284ae645a32264b0118 not found: ID does not exist"
Sep 04 23:02:32.666561 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.666509    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:32.702458 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.702431    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lswkp\" (UniqueName: \"kubernetes.io/projected/2fff9f17-de3f-46f0-801a-032b10d25e06-kube-api-access-lswkp\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:02:32.702458 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.702455    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fff9f17-de3f-46f0-801a-032b10d25e06-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:02:32.792093 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.792016    2578 patch_prober.go:28] interesting pod/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 04 23:02:32.792093 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.792070    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 04 23:02:32.918353 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.918318    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"]
Sep 04 23:02:32.921520 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:32.921484    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e8480dce-58489f8db9-2ssd7"]
Sep 04 23:02:33.539118 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:33.539067    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:33.792575 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:33.792484    2578 patch_prober.go:28] interesting pod/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 04 23:02:33.792575 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:33.792541    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 04 23:02:33.977696 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:33.977659    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:34.272339 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.272304    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"]
Sep 04 23:02:34.273195 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.273164    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="353120ad-a9d1-4142-afe1-d64f8d6a6ea6" containerName="revoke-keys"
Sep 04 23:02:34.273195 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.273191    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="353120ad-a9d1-4142-afe1-d64f8d6a6ea6" containerName="revoke-keys"
Sep 04 23:02:34.273381 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.273239    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fff9f17-de3f-46f0-801a-032b10d25e06" containerName="payload-processing"
Sep 04 23:02:34.273381 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.273248    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fff9f17-de3f-46f0-801a-032b10d25e06" containerName="payload-processing"
Sep 04 23:02:34.273381 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.273339    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fff9f17-de3f-46f0-801a-032b10d25e06" containerName="payload-processing"
Sep 04 23:02:34.273381 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.273353    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="353120ad-a9d1-4142-afe1-d64f8d6a6ea6" containerName="revoke-keys"
Sep 04 23:02:34.278233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.278213    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.281053 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.280966    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-8e65e509-openshift-default-dockercfg-zvfcn\""
Sep 04 23:02:34.287838 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.287813    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"]
Sep 04 23:02:34.308182 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.308152    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:34.410212 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.410164    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:34.420665 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.420594    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-envoy\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.420665 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.420595    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fff9f17-de3f-46f0-801a-032b10d25e06" path="/var/lib/kubelet/pods/2fff9f17-de3f-46f0-801a-032b10d25e06/volumes"
Sep 04 23:02:34.420906 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.420677    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-credential-socket\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.420906 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.420717    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-podinfo\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.420906 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.420760    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-data\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.420906 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.420792    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tthc6\" (UniqueName: \"kubernetes.io/projected/4a9edb7b-3f1b-428c-af30-e09421930b19-kube-api-access-tthc6\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.420906 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.420853    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-workload-socket\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.420906 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.420888    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-workload-certs\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.421224 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.420922    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-token\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.421224 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.420949    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9edb7b-3f1b-428c-af30-e09421930b19-istiod-ca-cert\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.521957 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.521918    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-data\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.521957 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.521961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tthc6\" (UniqueName: \"kubernetes.io/projected/4a9edb7b-3f1b-428c-af30-e09421930b19-kube-api-access-tthc6\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.522215 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.522014    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-workload-socket\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.522215 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.522046    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-workload-certs\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.522215 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.522080    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-token\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.522215 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.522105    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9edb7b-3f1b-428c-af30-e09421930b19-istiod-ca-cert\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.522468 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.522236    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-envoy\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.522468 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.522277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-credential-socket\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.522468 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.522311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-podinfo\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.522468 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.522384    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-data\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.522468 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.522445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-workload-socket\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.522787 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.522500    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-workload-certs\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.522787 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.522744    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-credential-socket\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.523077 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.523054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9edb7b-3f1b-428c-af30-e09421930b19-istiod-ca-cert\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.524594 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.524576    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-envoy\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.524842 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.524822    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-podinfo\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.530557 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.530533    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-token\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.530805 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.530782    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tthc6\" (UniqueName: \"kubernetes.io/projected/4a9edb7b-3f1b-428c-af30-e09421930b19-kube-api-access-tthc6\") pod \"e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") " pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.587732 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.587694    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:34.592690 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.592635    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:34.875380 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.875347    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"]
Sep 04 23:02:34.878988 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:02:34.878962    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9edb7b_3f1b_428c_af30_e09421930b19.slice/crio-899cd17b25183a5c5351949ae01d981ab0f59c6e3cca71c8553a7473720e1965 WatchSource:0}: Error finding container 899cd17b25183a5c5351949ae01d981ab0f59c6e3cca71c8553a7473720e1965: Status 404 returned error can't find the container with id 899cd17b25183a5c5351949ae01d981ab0f59c6e3cca71c8553a7473720e1965
Sep 04 23:02:34.881976 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.881938    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:02:34.882093 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.882023    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:02:34.882093 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.882065    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:02:34.909375 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.909351    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:34.909686 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.909666    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:34.910821 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:34.910798    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:02:35.618305 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:35.618266    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" event={"ID":"4a9edb7b-3f1b-428c-af30-e09421930b19","Type":"ContainerStarted","Data":"4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d"}
Sep 04 23:02:35.618735 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:35.618335    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" event={"ID":"4a9edb7b-3f1b-428c-af30-e09421930b19","Type":"ContainerStarted","Data":"899cd17b25183a5c5351949ae01d981ab0f59c6e3cca71c8553a7473720e1965"}
Sep 04 23:02:35.639879 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:35.639835    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" podStartSLOduration=1.639820908 podStartE2EDuration="1.639820908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:02:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:02:35.636973234 +0000 UTC m=+1043.747093824" watchObservedRunningTime="2026-09-04 23:02:35.639820908 +0000 UTC m=+1043.749941535"
Sep 04 23:02:35.983384 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:35.983292    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:36.593891 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:36.593851    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:36.599015 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:36.598988    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:36.625716 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:36.625675    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:36.626551 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:36.626530    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:02:36.981337 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:36.981258    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:37.073036 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:37.073004    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:38.131376 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:38.131345    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:38.483677 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:38.483571    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:38.583553 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:38.583517    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:38.987308 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:38.987277    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:39.045436 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:39.045402    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:44.026475 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:44.026437    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:55.432378 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:55.432346    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:02:59.081113 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:02:59.081076    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:00.786184 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:00.786146    2578 generic.go:356] "Generic (PLEG): container finished" podID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerID="4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d" exitCode=0
Sep 04 23:03:00.786741 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:00.786224    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" event={"ID":"4a9edb7b-3f1b-428c-af30-e09421930b19","Type":"ContainerDied","Data":"4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d"}
Sep 04 23:03:00.786823 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:00.786764    2578 scope.go:117] "RemoveContainer" containerID="4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d"
Sep 04 23:03:00.788102 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:00.788073    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:03:00.788191 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:00.788173    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:03:00.788236 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:00.788209    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:03:01.794192 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:01.794160    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" event={"ID":"4a9edb7b-3f1b-428c-af30-e09421930b19","Type":"ContainerStarted","Data":"0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a"}
Sep 04 23:03:01.808520 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:01.808488    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh"]
Sep 04 23:03:01.812179 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:01.812157    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh"
Sep 04 23:03:01.814517 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:01.814496    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:03:01.823689 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:01.823663    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh"]
Sep 04 23:03:01.880565 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:01.880518    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29104e2-1158-47d9-ba68-3a3c5d0b92d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh\" (UID: \"d29104e2-1158-47d9-ba68-3a3c5d0b92d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh"
Sep 04 23:03:01.981571 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:01.981527    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29104e2-1158-47d9-ba68-3a3c5d0b92d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh\" (UID: \"d29104e2-1158-47d9-ba68-3a3c5d0b92d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh"
Sep 04 23:03:01.982158 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:01.982135    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29104e2-1158-47d9-ba68-3a3c5d0b92d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh\" (UID: \"d29104e2-1158-47d9-ba68-3a3c5d0b92d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh"
Sep 04 23:03:02.125223 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:02.125187    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh"
Sep 04 23:03:02.421924 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:02.421896    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh"]
Sep 04 23:03:02.422364 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:03:02.422342    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29104e2_1158_47d9_ba68_3a3c5d0b92d5.slice/crio-86ad8720343ab0ad072bf4a1b9685c62693c0103c9f99ffcbf391782d2b7f3de WatchSource:0}: Error finding container 86ad8720343ab0ad072bf4a1b9685c62693c0103c9f99ffcbf391782d2b7f3de: Status 404 returned error can't find the container with id 86ad8720343ab0ad072bf4a1b9685c62693c0103c9f99ffcbf391782d2b7f3de
Sep 04 23:03:02.593528 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:02.593487    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:03:02.594746 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:02.594716    2578 patch_prober.go:28] interesting pod/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 04 23:03:02.594875 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:02.594771    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 04 23:03:02.801446 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:02.801360    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh" event={"ID":"d29104e2-1158-47d9-ba68-3a3c5d0b92d5","Type":"ContainerStarted","Data":"86ad8720343ab0ad072bf4a1b9685c62693c0103c9f99ffcbf391782d2b7f3de"}
Sep 04 23:03:03.593623 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:03.593585    2578 patch_prober.go:28] interesting pod/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 04 23:03:03.593815 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:03.593664    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 04 23:03:03.809307 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:03.809212    2578 generic.go:356] "Generic (PLEG): container finished" podID="d29104e2-1158-47d9-ba68-3a3c5d0b92d5" containerID="c218cc61104221f213a744cb7a674263f8807700c8607152a27bc55d8bc2f9da" exitCode=0
Sep 04 23:03:03.809307 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:03.809285    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh" event={"ID":"d29104e2-1158-47d9-ba68-3a3c5d0b92d5","Type":"ContainerDied","Data":"c218cc61104221f213a744cb7a674263f8807700c8607152a27bc55d8bc2f9da"}
Sep 04 23:03:04.593447 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:04.593412    2578 patch_prober.go:28] interesting pod/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 04 23:03:04.593627 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:04.593473    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 04 23:03:04.989348 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:04.989322    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh"
Sep 04 23:03:05.111314 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:05.111276    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29104e2-1158-47d9-ba68-3a3c5d0b92d5-maas-api-service-ca\") pod \"d29104e2-1158-47d9-ba68-3a3c5d0b92d5\" (UID: \"d29104e2-1158-47d9-ba68-3a3c5d0b92d5\") "
Sep 04 23:03:05.111633 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:05.111613    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d29104e2-1158-47d9-ba68-3a3c5d0b92d5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d29104e2-1158-47d9-ba68-3a3c5d0b92d5" (UID: "d29104e2-1158-47d9-ba68-3a3c5d0b92d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:03:05.212943 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:05.212852    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29104e2-1158-47d9-ba68-3a3c5d0b92d5-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:05.593492 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:05.593456    2578 patch_prober.go:28] interesting pod/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 04 23:03:05.593688 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:05.593518    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 04 23:03:05.822163 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:05.822131    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh"
Sep 04 23:03:05.822355 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:05.822328    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh" event={"ID":"d29104e2-1158-47d9-ba68-3a3c5d0b92d5","Type":"ContainerDied","Data":"86ad8720343ab0ad072bf4a1b9685c62693c0103c9f99ffcbf391782d2b7f3de"}
Sep 04 23:03:05.822480 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:05.822467    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86ad8720343ab0ad072bf4a1b9685c62693c0103c9f99ffcbf391782d2b7f3de"
Sep 04 23:03:06.593284 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.593245    2578 patch_prober.go:28] interesting pod/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 04 23:03:06.593687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.593303    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 04 23:03:06.625785 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.625752    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:03:06.647362 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.647333    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:06.751397 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.751359    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n"]
Sep 04 23:03:06.752072 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.752037    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d29104e2-1158-47d9-ba68-3a3c5d0b92d5" containerName="revoke-keys"
Sep 04 23:03:06.752072 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.752065    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d29104e2-1158-47d9-ba68-3a3c5d0b92d5" containerName="revoke-keys"
Sep 04 23:03:06.752269 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.752162    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d29104e2-1158-47d9-ba68-3a3c5d0b92d5" containerName="revoke-keys"
Sep 04 23:03:06.762565 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.762524    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n"]
Sep 04 23:03:06.762800 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.762632    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n"
Sep 04 23:03:06.765283 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.765258    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:03:06.830094 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.830063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b79c3b71-34c3-4fb7-a650-18fe78227f3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-72563d-vsn8n\" (UID: \"b79c3b71-34c3-4fb7-a650-18fe78227f3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n"
Sep 04 23:03:06.856737 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.856634    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:06.931251 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.931218    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b79c3b71-34c3-4fb7-a650-18fe78227f3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-72563d-vsn8n\" (UID: \"b79c3b71-34c3-4fb7-a650-18fe78227f3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n"
Sep 04 23:03:06.931917 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:06.931892    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b79c3b71-34c3-4fb7-a650-18fe78227f3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-72563d-vsn8n\" (UID: \"b79c3b71-34c3-4fb7-a650-18fe78227f3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n"
Sep 04 23:03:07.078355 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:07.078323    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n"
Sep 04 23:03:07.348741 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:07.348704    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n"]
Sep 04 23:03:07.349585 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:03:07.349550    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb79c3b71_34c3_4fb7_a650_18fe78227f3c.slice/crio-3b4ab869811850088093a90086d1195a1850f5a4068d653c978744e401768ccc WatchSource:0}: Error finding container 3b4ab869811850088093a90086d1195a1850f5a4068d653c978744e401768ccc: Status 404 returned error can't find the container with id 3b4ab869811850088093a90086d1195a1850f5a4068d653c978744e401768ccc
Sep 04 23:03:07.597376 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:07.597347    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:03:07.598403 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:07.598382    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:03:07.836585 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:07.836541    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n" event={"ID":"b79c3b71-34c3-4fb7-a650-18fe78227f3c","Type":"ContainerStarted","Data":"3961eec30a2d8d798ca3a231759da282d40cdef5b0b37c88f03527c15a9225f0"}
Sep 04 23:03:07.836807 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:07.836615    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n" event={"ID":"b79c3b71-34c3-4fb7-a650-18fe78227f3c","Type":"ContainerStarted","Data":"3b4ab869811850088093a90086d1195a1850f5a4068d653c978744e401768ccc"}
Sep 04 23:03:07.853301 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:07.853244    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n" podStartSLOduration=1.853229842 podStartE2EDuration="1.853229842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:03:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:03:07.850539171 +0000 UTC m=+1075.960659757" watchObservedRunningTime="2026-09-04 23:03:07.853229842 +0000 UTC m=+1075.963350428"
Sep 04 23:03:08.307408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:08.307375    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:08.844359 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:08.844325    2578 generic.go:356] "Generic (PLEG): container finished" podID="b79c3b71-34c3-4fb7-a650-18fe78227f3c" containerID="3961eec30a2d8d798ca3a231759da282d40cdef5b0b37c88f03527c15a9225f0" exitCode=0
Sep 04 23:03:08.844767 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:08.844408    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n" event={"ID":"b79c3b71-34c3-4fb7-a650-18fe78227f3c","Type":"ContainerDied","Data":"3961eec30a2d8d798ca3a231759da282d40cdef5b0b37c88f03527c15a9225f0"}
Sep 04 23:03:09.007565 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:09.007521    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:09.108835 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:09.108749    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:09.199515 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:09.199481    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh"]
Sep 04 23:03:09.200818 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:09.200787    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:09.202405 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:09.202386    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e65e509-pfqxh"]
Sep 04 23:03:09.264967 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:09.264935    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:09.332114 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:09.332075    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"]
Sep 04 23:03:09.332314 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:09.332285    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerName="istio-proxy" containerID="cri-o://0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a" gracePeriod=30
Sep 04 23:03:09.346065 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:09.346017    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:09.995451 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:09.995371    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n"
Sep 04 23:03:10.054921 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:10.054886    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:10.061905 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:10.061875    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b79c3b71-34c3-4fb7-a650-18fe78227f3c-maas-api-service-ca\") pod \"b79c3b71-34c3-4fb7-a650-18fe78227f3c\" (UID: \"b79c3b71-34c3-4fb7-a650-18fe78227f3c\") "
Sep 04 23:03:10.062269 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:10.062234    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b79c3b71-34c3-4fb7-a650-18fe78227f3c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b79c3b71-34c3-4fb7-a650-18fe78227f3c" (UID: "b79c3b71-34c3-4fb7-a650-18fe78227f3c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:03:10.150983 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:10.150907    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:10.163686 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:10.163631    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b79c3b71-34c3-4fb7-a650-18fe78227f3c-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:10.253334 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:10.253301    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:10.420197 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:10.420111    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d29104e2-1158-47d9-ba68-3a3c5d0b92d5" path="/var/lib/kubelet/pods/d29104e2-1158-47d9-ba68-3a3c5d0b92d5/volumes"
Sep 04 23:03:10.859213 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:10.859179    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n"
Sep 04 23:03:10.859409 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:10.859182    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n" event={"ID":"b79c3b71-34c3-4fb7-a650-18fe78227f3c","Type":"ContainerDied","Data":"3b4ab869811850088093a90086d1195a1850f5a4068d653c978744e401768ccc"}
Sep 04 23:03:10.859409 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:10.859299    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b4ab869811850088093a90086d1195a1850f5a4068d653c978744e401768ccc"
Sep 04 23:03:11.678711 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:11.678680    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:14.598300 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.598240    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:03:14.704934 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.704899    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9edb7b-3f1b-428c-af30-e09421930b19-istiod-ca-cert\") pod \"4a9edb7b-3f1b-428c-af30-e09421930b19\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") "
Sep 04 23:03:14.705098 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.704949    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-workload-certs\") pod \"4a9edb7b-3f1b-428c-af30-e09421930b19\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") "
Sep 04 23:03:14.705098 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.704978    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-data\") pod \"4a9edb7b-3f1b-428c-af30-e09421930b19\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") "
Sep 04 23:03:14.705098 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705036    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-credential-socket\") pod \"4a9edb7b-3f1b-428c-af30-e09421930b19\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") "
Sep 04 23:03:14.705098 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705061    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-envoy\") pod \"4a9edb7b-3f1b-428c-af30-e09421930b19\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") "
Sep 04 23:03:14.705320 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705122    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-podinfo\") pod \"4a9edb7b-3f1b-428c-af30-e09421930b19\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") "
Sep 04 23:03:14.705320 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705187    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tthc6\" (UniqueName: \"kubernetes.io/projected/4a9edb7b-3f1b-428c-af30-e09421930b19-kube-api-access-tthc6\") pod \"4a9edb7b-3f1b-428c-af30-e09421930b19\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") "
Sep 04 23:03:14.705320 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705222    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-workload-socket\") pod \"4a9edb7b-3f1b-428c-af30-e09421930b19\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") "
Sep 04 23:03:14.705320 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705257    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-token\") pod \"4a9edb7b-3f1b-428c-af30-e09421930b19\" (UID: \"4a9edb7b-3f1b-428c-af30-e09421930b19\") "
Sep 04 23:03:14.705518 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705313    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4a9edb7b-3f1b-428c-af30-e09421930b19" (UID: "4a9edb7b-3f1b-428c-af30-e09421930b19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:03:14.705518 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705373    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9edb7b-3f1b-428c-af30-e09421930b19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4a9edb7b-3f1b-428c-af30-e09421930b19" (UID: "4a9edb7b-3f1b-428c-af30-e09421930b19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:03:14.705518 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705323    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-data" (OuterVolumeSpecName: "istio-data") pod "4a9edb7b-3f1b-428c-af30-e09421930b19" (UID: "4a9edb7b-3f1b-428c-af30-e09421930b19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:03:14.705518 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705391    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4a9edb7b-3f1b-428c-af30-e09421930b19" (UID: "4a9edb7b-3f1b-428c-af30-e09421930b19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:03:14.705767 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705593    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4a9edb7b-3f1b-428c-af30-e09421930b19" (UID: "4a9edb7b-3f1b-428c-af30-e09421930b19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:03:14.705767 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705701    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:14.705767 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705722    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9edb7b-3f1b-428c-af30-e09421930b19-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:14.705767 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705736    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:14.705767 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705748    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:14.705767 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.705760    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:14.707454 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.707423    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a9edb7b-3f1b-428c-af30-e09421930b19-kube-api-access-tthc6" (OuterVolumeSpecName: "kube-api-access-tthc6") pod "4a9edb7b-3f1b-428c-af30-e09421930b19" (UID: "4a9edb7b-3f1b-428c-af30-e09421930b19"). InnerVolumeSpecName "kube-api-access-tthc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:03:14.707454 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.707421    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4a9edb7b-3f1b-428c-af30-e09421930b19" (UID: "4a9edb7b-3f1b-428c-af30-e09421930b19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:03:14.707688 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.707672    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-token" (OuterVolumeSpecName: "istio-token") pod "4a9edb7b-3f1b-428c-af30-e09421930b19" (UID: "4a9edb7b-3f1b-428c-af30-e09421930b19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:03:14.707750 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.707692    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4a9edb7b-3f1b-428c-af30-e09421930b19" (UID: "4a9edb7b-3f1b-428c-af30-e09421930b19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:03:14.806408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.806367    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tthc6\" (UniqueName: \"kubernetes.io/projected/4a9edb7b-3f1b-428c-af30-e09421930b19-kube-api-access-tthc6\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:14.806408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.806402    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:14.806408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.806413    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:14.806680 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.806421    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9edb7b-3f1b-428c-af30-e09421930b19-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:14.885406 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.885321    2578 generic.go:356] "Generic (PLEG): container finished" podID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerID="0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a" exitCode=0
Sep 04 23:03:14.885406 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.885370    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"
Sep 04 23:03:14.885584 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.885412    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" event={"ID":"4a9edb7b-3f1b-428c-af30-e09421930b19","Type":"ContainerDied","Data":"0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a"}
Sep 04 23:03:14.885584 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.885472    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c" event={"ID":"4a9edb7b-3f1b-428c-af30-e09421930b19","Type":"ContainerDied","Data":"899cd17b25183a5c5351949ae01d981ab0f59c6e3cca71c8553a7473720e1965"}
Sep 04 23:03:14.885584 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.885497    2578 scope.go:117] "RemoveContainer" containerID="0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a"
Sep 04 23:03:14.895252 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.895233    2578 scope.go:117] "RemoveContainer" containerID="4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d"
Sep 04 23:03:14.903470 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.903449    2578 scope.go:117] "RemoveContainer" containerID="0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a"
Sep 04 23:03:14.903748 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:03:14.903728    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a\": container with ID starting with 0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a not found: ID does not exist" containerID="0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a"
Sep 04 23:03:14.903819 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.903756    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a"} err="failed to get container status \"0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a\": rpc error: code = NotFound desc = could not find container \"0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a\": container with ID starting with 0f0e36b6bb8a552c424a30e47ca136e8ba4d800c79b04bca36bb70fb90f8825a not found: ID does not exist"
Sep 04 23:03:14.903819 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.903773    2578 scope.go:117] "RemoveContainer" containerID="4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d"
Sep 04 23:03:14.904039 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:03:14.904022    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d\": container with ID starting with 4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d not found: ID does not exist" containerID="4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d"
Sep 04 23:03:14.904080 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.904045    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d"} err="failed to get container status \"4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d\": rpc error: code = NotFound desc = could not find container \"4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d\": container with ID starting with 4dc403aedafb871f01c2fe96114adf66fad06f4260d2d150f9674e3bd73bd36d not found: ID does not exist"
Sep 04 23:03:14.909880 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.909856    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"]
Sep 04 23:03:14.913687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:14.913665    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-8e65e509-openshift-default-7ccf4fd8f6-knf2c"]
Sep 04 23:03:16.420514 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:16.420482    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" path="/var/lib/kubelet/pods/4a9edb7b-3f1b-428c-af30-e09421930b19/volumes"
Sep 04 23:03:17.027650 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:17.027600    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:17.621773 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:17.621745    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:17.690835 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:17.690801    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:17.751406 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:17.751374    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:17.823629 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:17.823596    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n"]
Sep 04 23:03:17.826314 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:17.826284    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:17.827326 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:17.827305    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-72563d-vsn8n"]
Sep 04 23:03:17.892844 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:17.892730    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:18.200437 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:18.200340    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:18.371871 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:18.371837    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:18.419732 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:18.419696    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b79c3b71-34c3-4fb7-a650-18fe78227f3c" path="/var/lib/kubelet/pods/b79c3b71-34c3-4fb7-a650-18fe78227f3c/volumes"
Sep 04 23:03:18.442998 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:18.442968    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:18.523763 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:18.523672    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:18.623360 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:18.623326    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:18.703885 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:18.703846    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:19.110466 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:19.110429    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:23.272163 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:23.272125    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:28.964267 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:28.964234    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:29.220909 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:29.220835    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"]
Sep 04 23:03:29.221077 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:29.221054    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerName="istio-proxy" containerID="cri-o://2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd" gracePeriod=30
Sep 04 23:03:29.242632 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:29.242606    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:29.704516 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:29.704484    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:30.044298 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:30.044208    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:30.477526 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:30.477488    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:31.027774 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:31.027743    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:31.133571 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:31.133532    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:31.264944 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:31.264911    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:34.479279 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.479253    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:03:34.497127 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497056    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-credential-socket\") pod \"6974e567-2f6c-468a-9a35-64fc79e91e61\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") "
Sep 04 23:03:34.497127 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497102    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-envoy\") pod \"6974e567-2f6c-468a-9a35-64fc79e91e61\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") "
Sep 04 23:03:34.497333 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497140    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-workload-socket\") pod \"6974e567-2f6c-468a-9a35-64fc79e91e61\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") "
Sep 04 23:03:34.497333 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497232    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lllm\" (UniqueName: \"kubernetes.io/projected/6974e567-2f6c-468a-9a35-64fc79e91e61-kube-api-access-6lllm\") pod \"6974e567-2f6c-468a-9a35-64fc79e91e61\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") "
Sep 04 23:03:34.497333 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497268    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-podinfo\") pod \"6974e567-2f6c-468a-9a35-64fc79e91e61\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") "
Sep 04 23:03:34.497510 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497346    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6974e567-2f6c-468a-9a35-64fc79e91e61" (UID: "6974e567-2f6c-468a-9a35-64fc79e91e61"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:03:34.497676 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497633    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6974e567-2f6c-468a-9a35-64fc79e91e61" (UID: "6974e567-2f6c-468a-9a35-64fc79e91e61"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:03:34.497752 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497460    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-data\") pod \"6974e567-2f6c-468a-9a35-64fc79e91e61\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") "
Sep 04 23:03:34.497752 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497722    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-workload-certs\") pod \"6974e567-2f6c-468a-9a35-64fc79e91e61\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") "
Sep 04 23:03:34.497860 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497782    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6974e567-2f6c-468a-9a35-64fc79e91e61-istiod-ca-cert\") pod \"6974e567-2f6c-468a-9a35-64fc79e91e61\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") "
Sep 04 23:03:34.497860 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497793    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-data" (OuterVolumeSpecName: "istio-data") pod "6974e567-2f6c-468a-9a35-64fc79e91e61" (UID: "6974e567-2f6c-468a-9a35-64fc79e91e61"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:03:34.497860 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497822    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-token\") pod \"6974e567-2f6c-468a-9a35-64fc79e91e61\" (UID: \"6974e567-2f6c-468a-9a35-64fc79e91e61\") "
Sep 04 23:03:34.498010 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.497925    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6974e567-2f6c-468a-9a35-64fc79e91e61" (UID: "6974e567-2f6c-468a-9a35-64fc79e91e61"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:03:34.498397 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.498124    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6974e567-2f6c-468a-9a35-64fc79e91e61-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6974e567-2f6c-468a-9a35-64fc79e91e61" (UID: "6974e567-2f6c-468a-9a35-64fc79e91e61"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:03:34.498397 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.498252    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:34.498397 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.498272    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:34.498397 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.498288    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:34.498397 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.498302    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:34.498397 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.498316    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6974e567-2f6c-468a-9a35-64fc79e91e61-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:34.501256 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.501035    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6974e567-2f6c-468a-9a35-64fc79e91e61" (UID: "6974e567-2f6c-468a-9a35-64fc79e91e61"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:03:34.501359 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.501261    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6974e567-2f6c-468a-9a35-64fc79e91e61-kube-api-access-6lllm" (OuterVolumeSpecName: "kube-api-access-6lllm") pod "6974e567-2f6c-468a-9a35-64fc79e91e61" (UID: "6974e567-2f6c-468a-9a35-64fc79e91e61"). InnerVolumeSpecName "kube-api-access-6lllm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:03:34.501419 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.501382    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6974e567-2f6c-468a-9a35-64fc79e91e61" (UID: "6974e567-2f6c-468a-9a35-64fc79e91e61"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:03:34.501925 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.501899    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-token" (OuterVolumeSpecName: "istio-token") pod "6974e567-2f6c-468a-9a35-64fc79e91e61" (UID: "6974e567-2f6c-468a-9a35-64fc79e91e61"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:03:34.599002 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.598971    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lllm\" (UniqueName: \"kubernetes.io/projected/6974e567-2f6c-468a-9a35-64fc79e91e61-kube-api-access-6lllm\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:34.599002 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.599001    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:34.599194 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.599015    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:34.599194 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:34.599026    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6974e567-2f6c-468a-9a35-64fc79e91e61-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:03:35.010949 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:35.010907    2578 generic.go:356] "Generic (PLEG): container finished" podID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerID="2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd" exitCode=0
Sep 04 23:03:35.010949 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:35.010951    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" event={"ID":"6974e567-2f6c-468a-9a35-64fc79e91e61","Type":"ContainerDied","Data":"2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd"}
Sep 04 23:03:35.011163 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:35.010994    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg" event={"ID":"6974e567-2f6c-468a-9a35-64fc79e91e61","Type":"ContainerDied","Data":"c950a1cf8dee9a91db6fa80e695213b7fc7c621f8f8372d8a27161998eb84b8f"}
Sep 04 23:03:35.011163 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:35.011015    2578 scope.go:117] "RemoveContainer" containerID="2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd"
Sep 04 23:03:35.011163 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:35.010952    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"
Sep 04 23:03:35.021294 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:35.021274    2578 scope.go:117] "RemoveContainer" containerID="3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a"
Sep 04 23:03:35.029688 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:35.029667    2578 scope.go:117] "RemoveContainer" containerID="2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd"
Sep 04 23:03:35.029974 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:03:35.029954    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd\": container with ID starting with 2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd not found: ID does not exist" containerID="2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd"
Sep 04 23:03:35.030020 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:35.029982    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd"} err="failed to get container status \"2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd\": rpc error: code = NotFound desc = could not find container \"2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd\": container with ID starting with 2045f5d335ecf2df2f05067f15e2d27bfdc81cdaf043737030bc556a095cc3dd not found: ID does not exist"
Sep 04 23:03:35.030020 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:35.029999    2578 scope.go:117] "RemoveContainer" containerID="3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a"
Sep 04 23:03:35.030252 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:03:35.030234    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a\": container with ID starting with 3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a not found: ID does not exist" containerID="3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a"
Sep 04 23:03:35.030293 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:35.030258    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a"} err="failed to get container status \"3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a\": rpc error: code = NotFound desc = could not find container \"3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a\": container with ID starting with 3297470709b255cb136fa438f550a872a6bc35492d219d1c569a96488597013a not found: ID does not exist"
Sep 04 23:03:35.036983 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:35.036958    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"]
Sep 04 23:03:35.040900 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:35.040878    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-72563d-openshift-default-65fcf874bd-dbrhg"]
Sep 04 23:03:36.419883 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:36.419848    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" path="/var/lib/kubelet/pods/6974e567-2f6c-468a-9a35-64fc79e91e61/volumes"
Sep 04 23:03:39.284082 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:39.284049    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:42.980473 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:42.980440    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:43.035847 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:43.035815    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:45.037372 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:45.037334    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:51.145312 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:51.145283    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:51.251846 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:51.251812    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:51.351462 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:51.351401    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:52.651961 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:52.651907    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:52.784902 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:52.784869    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:53.021154 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:53.021033    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:53.141026 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:53.140990    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:58.166136 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:58.166049    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:03:59.274193 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:59.274153    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"]
Sep 04 23:03:59.296014 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:03:59.295961    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:00.233102 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:00.233068    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:00.522542 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:00.522460    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:07.204128 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:07.204087    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:10.813663 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:10.813605    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:10.838927 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:10.838896    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"]
Sep 04 23:04:10.839158 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:10.839133    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4" podUID="40c1e521-9ffa-4dc1-ac59-1e4e9226202e" containerName="payload-processing" containerID="cri-o://f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2" gracePeriod=30
Sep 04 23:04:10.848712 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:10.848683    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"]
Sep 04 23:04:10.848935 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:10.848911    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp" podUID="8d8fbd59-9084-4693-a2c0-1b1ba5d43185" containerName="payload-pre-processing" containerID="cri-o://965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8" gracePeriod=30
Sep 04 23:04:10.884619 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:10.884589    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:10.949951 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:10.949923    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:11.012772 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.012739    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:11.078075 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.077839    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:11.140224 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.140198    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"
Sep 04 23:04:11.165177 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.165154    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"
Sep 04 23:04:11.238021 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.237985    2578 generic.go:356] "Generic (PLEG): container finished" podID="40c1e521-9ffa-4dc1-ac59-1e4e9226202e" containerID="f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2" exitCode=0
Sep 04 23:04:11.238204 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.238031    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"
Sep 04 23:04:11.238204 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.238064    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4" event={"ID":"40c1e521-9ffa-4dc1-ac59-1e4e9226202e","Type":"ContainerDied","Data":"f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2"}
Sep 04 23:04:11.238204 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.238117    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4" event={"ID":"40c1e521-9ffa-4dc1-ac59-1e4e9226202e","Type":"ContainerDied","Data":"d36fd63b6d5ee829118c11f0a5c8322c74f9ddb1e3e9baf39c11b0ad6ffdb661"}
Sep 04 23:04:11.238204 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.238141    2578 scope.go:117] "RemoveContainer" containerID="f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2"
Sep 04 23:04:11.239547 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.239525    2578 generic.go:356] "Generic (PLEG): container finished" podID="8d8fbd59-9084-4693-a2c0-1b1ba5d43185" containerID="965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8" exitCode=0
Sep 04 23:04:11.239652 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.239555    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"
Sep 04 23:04:11.239652 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.239588    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp" event={"ID":"8d8fbd59-9084-4693-a2c0-1b1ba5d43185","Type":"ContainerDied","Data":"965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8"}
Sep 04 23:04:11.239652 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.239627    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp" event={"ID":"8d8fbd59-9084-4693-a2c0-1b1ba5d43185","Type":"ContainerDied","Data":"e5b9e208e57c5a47b94e5b70662bae329e30b0f38a34801dfee3f1678eb2be09"}
Sep 04 23:04:11.248330 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.248313    2578 scope.go:117] "RemoveContainer" containerID="f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2"
Sep 04 23:04:11.248608 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:04:11.248591    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2\": container with ID starting with f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2 not found: ID does not exist" containerID="f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2"
Sep 04 23:04:11.248716 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.248615    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2"} err="failed to get container status \"f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2\": rpc error: code = NotFound desc = could not find container \"f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2\": container with ID starting with f4b7554e542d947e548756526d3d795a4bb7dfc0b4f0d4d12a83d93d5b20c4a2 not found: ID does not exist"
Sep 04 23:04:11.248716 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.248630    2578 scope.go:117] "RemoveContainer" containerID="965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8"
Sep 04 23:04:11.255780 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.255760    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zhgw\" (UniqueName: \"kubernetes.io/projected/40c1e521-9ffa-4dc1-ac59-1e4e9226202e-kube-api-access-4zhgw\") pod \"40c1e521-9ffa-4dc1-ac59-1e4e9226202e\" (UID: \"40c1e521-9ffa-4dc1-ac59-1e4e9226202e\") "
Sep 04 23:04:11.255868 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.255802    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzn6n\" (UniqueName: \"kubernetes.io/projected/8d8fbd59-9084-4693-a2c0-1b1ba5d43185-kube-api-access-xzn6n\") pod \"8d8fbd59-9084-4693-a2c0-1b1ba5d43185\" (UID: \"8d8fbd59-9084-4693-a2c0-1b1ba5d43185\") "
Sep 04 23:04:11.255868 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.255847    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d8fbd59-9084-4693-a2c0-1b1ba5d43185-plugins-config-volume\") pod \"8d8fbd59-9084-4693-a2c0-1b1ba5d43185\" (UID: \"8d8fbd59-9084-4693-a2c0-1b1ba5d43185\") "
Sep 04 23:04:11.255962 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.255882    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40c1e521-9ffa-4dc1-ac59-1e4e9226202e-plugins-config-volume\") pod \"40c1e521-9ffa-4dc1-ac59-1e4e9226202e\" (UID: \"40c1e521-9ffa-4dc1-ac59-1e4e9226202e\") "
Sep 04 23:04:11.256287 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.256263    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d8fbd59-9084-4693-a2c0-1b1ba5d43185-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8d8fbd59-9084-4693-a2c0-1b1ba5d43185" (UID: "8d8fbd59-9084-4693-a2c0-1b1ba5d43185"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:04:11.256388 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.256364    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40c1e521-9ffa-4dc1-ac59-1e4e9226202e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40c1e521-9ffa-4dc1-ac59-1e4e9226202e" (UID: "40c1e521-9ffa-4dc1-ac59-1e4e9226202e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:04:11.256847 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.256826    2578 scope.go:117] "RemoveContainer" containerID="965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8"
Sep 04 23:04:11.257185 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:04:11.257165    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8\": container with ID starting with 965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8 not found: ID does not exist" containerID="965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8"
Sep 04 23:04:11.257240 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.257192    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8"} err="failed to get container status \"965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8\": rpc error: code = NotFound desc = could not find container \"965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8\": container with ID starting with 965e37587ff66088835b2b07f65c5c839d63c6a5be5c214241149ec13a2ddee8 not found: ID does not exist"
Sep 04 23:04:11.257813 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.257793    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c1e521-9ffa-4dc1-ac59-1e4e9226202e-kube-api-access-4zhgw" (OuterVolumeSpecName: "kube-api-access-4zhgw") pod "40c1e521-9ffa-4dc1-ac59-1e4e9226202e" (UID: "40c1e521-9ffa-4dc1-ac59-1e4e9226202e"). InnerVolumeSpecName "kube-api-access-4zhgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:04:11.258171 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.258152    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d8fbd59-9084-4693-a2c0-1b1ba5d43185-kube-api-access-xzn6n" (OuterVolumeSpecName: "kube-api-access-xzn6n") pod "8d8fbd59-9084-4693-a2c0-1b1ba5d43185" (UID: "8d8fbd59-9084-4693-a2c0-1b1ba5d43185"). InnerVolumeSpecName "kube-api-access-xzn6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:04:11.357120 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.357044    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zhgw\" (UniqueName: \"kubernetes.io/projected/40c1e521-9ffa-4dc1-ac59-1e4e9226202e-kube-api-access-4zhgw\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:11.357120 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.357072    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzn6n\" (UniqueName: \"kubernetes.io/projected/8d8fbd59-9084-4693-a2c0-1b1ba5d43185-kube-api-access-xzn6n\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:11.357120 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.357082    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d8fbd59-9084-4693-a2c0-1b1ba5d43185-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:11.357120 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.357090    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40c1e521-9ffa-4dc1-ac59-1e4e9226202e-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:11.376747 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.376723    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:11.564168 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.564133    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"]
Sep 04 23:04:11.570237 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.570214    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a64414-7c44c445f7-5zxh4"]
Sep 04 23:04:11.589164 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.589135    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"]
Sep 04 23:04:11.591312 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:11.591290    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a64414-5b5bc69b9d-nfjxp"]
Sep 04 23:04:12.421038 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:12.421006    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40c1e521-9ffa-4dc1-ac59-1e4e9226202e" path="/var/lib/kubelet/pods/40c1e521-9ffa-4dc1-ac59-1e4e9226202e/volumes"
Sep 04 23:04:12.421400 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:12.421391    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d8fbd59-9084-4693-a2c0-1b1ba5d43185" path="/var/lib/kubelet/pods/8d8fbd59-9084-4693-a2c0-1b1ba5d43185/volumes"
Sep 04 23:04:13.069563 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:13.069529    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:13.139850 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:13.139819    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:13.220689 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:13.220634    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:13.322329 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:13.322251    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:14.294804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.294765    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw" podUID="79dd3773-f5a3-4063-873a-ccd5192f13ba" containerName="main" containerID="cri-o://edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f" gracePeriod=45
Sep 04 23:04:14.554581 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.554515    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:04:14.584877 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.584850    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-tmp-dir\") pod \"79dd3773-f5a3-4063-873a-ccd5192f13ba\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") "
Sep 04 23:04:14.585012 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.584907    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-dshm\") pod \"79dd3773-f5a3-4063-873a-ccd5192f13ba\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") "
Sep 04 23:04:14.585012 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.584939    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79dd3773-f5a3-4063-873a-ccd5192f13ba-tls-certs\") pod \"79dd3773-f5a3-4063-873a-ccd5192f13ba\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") "
Sep 04 23:04:14.585012 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.584981    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjgzv\" (UniqueName: \"kubernetes.io/projected/79dd3773-f5a3-4063-873a-ccd5192f13ba-kube-api-access-tjgzv\") pod \"79dd3773-f5a3-4063-873a-ccd5192f13ba\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") "
Sep 04 23:04:14.585190 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.585072    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-model-cache\") pod \"79dd3773-f5a3-4063-873a-ccd5192f13ba\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") "
Sep 04 23:04:14.585190 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.585104    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-home\") pod \"79dd3773-f5a3-4063-873a-ccd5192f13ba\" (UID: \"79dd3773-f5a3-4063-873a-ccd5192f13ba\") "
Sep 04 23:04:14.585190 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.585125    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "79dd3773-f5a3-4063-873a-ccd5192f13ba" (UID: "79dd3773-f5a3-4063-873a-ccd5192f13ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:04:14.585426 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.585372    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-tmp-dir\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:14.585426 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.585366    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "79dd3773-f5a3-4063-873a-ccd5192f13ba" (UID: "79dd3773-f5a3-4063-873a-ccd5192f13ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:04:14.585426 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.585391    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-home" (OuterVolumeSpecName: "home") pod "79dd3773-f5a3-4063-873a-ccd5192f13ba" (UID: "79dd3773-f5a3-4063-873a-ccd5192f13ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:04:14.587125 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.587103    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-dshm" (OuterVolumeSpecName: "dshm") pod "79dd3773-f5a3-4063-873a-ccd5192f13ba" (UID: "79dd3773-f5a3-4063-873a-ccd5192f13ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:04:14.587233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.587175    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79dd3773-f5a3-4063-873a-ccd5192f13ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "79dd3773-f5a3-4063-873a-ccd5192f13ba" (UID: "79dd3773-f5a3-4063-873a-ccd5192f13ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:04:14.587421 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.587406    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79dd3773-f5a3-4063-873a-ccd5192f13ba-kube-api-access-tjgzv" (OuterVolumeSpecName: "kube-api-access-tjgzv") pod "79dd3773-f5a3-4063-873a-ccd5192f13ba" (UID: "79dd3773-f5a3-4063-873a-ccd5192f13ba"). InnerVolumeSpecName "kube-api-access-tjgzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:04:14.686248 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.686200    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-model-cache\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:14.686248 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.686242    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-home\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:14.686248 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.686252    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79dd3773-f5a3-4063-873a-ccd5192f13ba-dshm\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:14.686248 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.686260    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79dd3773-f5a3-4063-873a-ccd5192f13ba-tls-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:14.686512 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:14.686268    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjgzv\" (UniqueName: \"kubernetes.io/projected/79dd3773-f5a3-4063-873a-ccd5192f13ba-kube-api-access-tjgzv\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:15.266630 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:15.266594    2578 generic.go:356] "Generic (PLEG): container finished" podID="79dd3773-f5a3-4063-873a-ccd5192f13ba" containerID="edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f" exitCode=0
Sep 04 23:04:15.266848 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:15.266669    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"
Sep 04 23:04:15.266848 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:15.266671    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw" event={"ID":"79dd3773-f5a3-4063-873a-ccd5192f13ba","Type":"ContainerDied","Data":"edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f"}
Sep 04 23:04:15.266848 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:15.266780    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw" event={"ID":"79dd3773-f5a3-4063-873a-ccd5192f13ba","Type":"ContainerDied","Data":"0072d332ea31f938de5e900f7b2f2849145c784f11c3231478197c1aa4396392"}
Sep 04 23:04:15.266848 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:15.266800    2578 scope.go:117] "RemoveContainer" containerID="edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f"
Sep 04 23:04:15.276580 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:15.276560    2578 scope.go:117] "RemoveContainer" containerID="edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f"
Sep 04 23:04:15.276856 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:04:15.276834    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f\": container with ID starting with edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f not found: ID does not exist" containerID="edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f"
Sep 04 23:04:15.276927 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:15.276868    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f"} err="failed to get container status \"edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f\": rpc error: code = NotFound desc = could not find container \"edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f\": container with ID starting with edee2874fe3d5af0ef05c2b8c7b3d3c2aa37358c05c3a5f2c0bb81af7083545f not found: ID does not exist"
Sep 04 23:04:15.288919 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:15.288893    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"]
Sep 04 23:04:15.291163 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:15.291145    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a64414/test-model-a64414-kserve-5b4b948f49-kn6fw"]
Sep 04 23:04:16.419605 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:16.419574    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79dd3773-f5a3-4063-873a-ccd5192f13ba" path="/var/lib/kubelet/pods/79dd3773-f5a3-4063-873a-ccd5192f13ba/volumes"
Sep 04 23:04:17.583267 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:17.583235    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:17.685345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:17.685314    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:17.904289 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:17.904208    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:18.209146 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209062    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h"]
Sep 04 23:04:18.209497 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209482    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d8fbd59-9084-4693-a2c0-1b1ba5d43185" containerName="payload-pre-processing"
Sep 04 23:04:18.209539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209501    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d8fbd59-9084-4693-a2c0-1b1ba5d43185" containerName="payload-pre-processing"
Sep 04 23:04:18.209539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209516    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerName="istio-proxy"
Sep 04 23:04:18.209539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209522    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerName="istio-proxy"
Sep 04 23:04:18.209539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209529    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerName="istio-proxy"
Sep 04 23:04:18.209539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209535    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerName="istio-proxy"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209545    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79dd3773-f5a3-4063-873a-ccd5192f13ba" containerName="main"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209550    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="79dd3773-f5a3-4063-873a-ccd5192f13ba" containerName="main"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209564    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b79c3b71-34c3-4fb7-a650-18fe78227f3c" containerName="revoke-keys"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209570    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b79c3b71-34c3-4fb7-a650-18fe78227f3c" containerName="revoke-keys"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209579    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40c1e521-9ffa-4dc1-ac59-1e4e9226202e" containerName="payload-processing"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209588    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="40c1e521-9ffa-4dc1-ac59-1e4e9226202e" containerName="payload-processing"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209667    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerName="istio-proxy"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209679    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerName="istio-proxy"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209689    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d8fbd59-9084-4693-a2c0-1b1ba5d43185" containerName="payload-pre-processing"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209699    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="40c1e521-9ffa-4dc1-ac59-1e4e9226202e" containerName="payload-processing"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209705    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b79c3b71-34c3-4fb7-a650-18fe78227f3c" containerName="revoke-keys"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209711    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="79dd3773-f5a3-4063-873a-ccd5192f13ba" containerName="main"
Sep 04 23:04:18.209727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.209716    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerName="istio-proxy"
Sep 04 23:04:18.214108 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.214091    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h"
Sep 04 23:04:18.216696 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.216670    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:04:18.220814 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.220788    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h"]
Sep 04 23:04:18.311048 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.311015    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:18.318111 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.318078    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed10899-9e21-44c7-8440-40a7e96b07a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h\" (UID: \"fed10899-9e21-44c7-8440-40a7e96b07a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h"
Sep 04 23:04:18.418936 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.418895    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed10899-9e21-44c7-8440-40a7e96b07a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h\" (UID: \"fed10899-9e21-44c7-8440-40a7e96b07a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h"
Sep 04 23:04:18.419521 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.419498    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed10899-9e21-44c7-8440-40a7e96b07a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h\" (UID: \"fed10899-9e21-44c7-8440-40a7e96b07a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h"
Sep 04 23:04:18.526447 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.526359    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h"
Sep 04 23:04:18.802959 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:18.802927    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h"]
Sep 04 23:04:18.804919 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:04:18.804889    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed10899_9e21_44c7_8440_40a7e96b07a0.slice/crio-bdfb7d3671834019397272f9d6ac7704790975b27a73c92fc1704e9b440bd88d WatchSource:0}: Error finding container bdfb7d3671834019397272f9d6ac7704790975b27a73c92fc1704e9b440bd88d: Status 404 returned error can't find the container with id bdfb7d3671834019397272f9d6ac7704790975b27a73c92fc1704e9b440bd88d
Sep 04 23:04:19.292164 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:19.292116    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h" event={"ID":"fed10899-9e21-44c7-8440-40a7e96b07a0","Type":"ContainerStarted","Data":"71e4b577f372de15e948602465b32221224903efc47378594ac55667ba3d3a5a"}
Sep 04 23:04:19.292358 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:19.292179    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h" event={"ID":"fed10899-9e21-44c7-8440-40a7e96b07a0","Type":"ContainerStarted","Data":"bdfb7d3671834019397272f9d6ac7704790975b27a73c92fc1704e9b440bd88d"}
Sep 04 23:04:19.306521 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:19.306464    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h" podStartSLOduration=1.306447047 podStartE2EDuration="1.306447047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:04:19.306113036 +0000 UTC m=+1147.416233622" watchObservedRunningTime="2026-09-04 23:04:19.306447047 +0000 UTC m=+1147.416567652"
Sep 04 23:04:20.299185 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:20.299151    2578 generic.go:356] "Generic (PLEG): container finished" podID="fed10899-9e21-44c7-8440-40a7e96b07a0" containerID="71e4b577f372de15e948602465b32221224903efc47378594ac55667ba3d3a5a" exitCode=0
Sep 04 23:04:20.299577 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:20.299243    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h" event={"ID":"fed10899-9e21-44c7-8440-40a7e96b07a0","Type":"ContainerDied","Data":"71e4b577f372de15e948602465b32221224903efc47378594ac55667ba3d3a5a"}
Sep 04 23:04:21.436064 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:21.436033    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h"
Sep 04 23:04:21.547130 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:21.547094    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed10899-9e21-44c7-8440-40a7e96b07a0-maas-api-service-ca\") pod \"fed10899-9e21-44c7-8440-40a7e96b07a0\" (UID: \"fed10899-9e21-44c7-8440-40a7e96b07a0\") "
Sep 04 23:04:21.547471 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:21.547450    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fed10899-9e21-44c7-8440-40a7e96b07a0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fed10899-9e21-44c7-8440-40a7e96b07a0" (UID: "fed10899-9e21-44c7-8440-40a7e96b07a0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:04:21.648632 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:21.648596    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed10899-9e21-44c7-8440-40a7e96b07a0-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:22.312472 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:22.312437    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h"
Sep 04 23:04:22.312738 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:22.312474    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h" event={"ID":"fed10899-9e21-44c7-8440-40a7e96b07a0","Type":"ContainerDied","Data":"bdfb7d3671834019397272f9d6ac7704790975b27a73c92fc1704e9b440bd88d"}
Sep 04 23:04:22.312738 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:22.312534    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdfb7d3671834019397272f9d6ac7704790975b27a73c92fc1704e9b440bd88d"
Sep 04 23:04:23.772805 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:23.772770    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:24.614871 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:24.614811    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:28.685183 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:28.685138    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:28.712513 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:28.712474    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"]
Sep 04 23:04:28.712724 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:28.712698    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr" podUID="4a4f352f-5591-4f2d-bdf7-129518a5f9cc" containerName="payload-processing" containerID="cri-o://399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f" gracePeriod=30
Sep 04 23:04:28.785530 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:28.785495    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:28.879962 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:28.879924    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h"]
Sep 04 23:04:28.881975 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:28.881947    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:28.882845 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:28.882825    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4fd6f6-4448h"]
Sep 04 23:04:28.947031 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:28.946940    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:28.984895 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:28.984868    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"
Sep 04 23:04:29.114176 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.114138    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwtjf\" (UniqueName: \"kubernetes.io/projected/4a4f352f-5591-4f2d-bdf7-129518a5f9cc-kube-api-access-pwtjf\") pod \"4a4f352f-5591-4f2d-bdf7-129518a5f9cc\" (UID: \"4a4f352f-5591-4f2d-bdf7-129518a5f9cc\") "
Sep 04 23:04:29.114378 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.114256    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4f352f-5591-4f2d-bdf7-129518a5f9cc-plugins-config-volume\") pod \"4a4f352f-5591-4f2d-bdf7-129518a5f9cc\" (UID: \"4a4f352f-5591-4f2d-bdf7-129518a5f9cc\") "
Sep 04 23:04:29.114664 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.114611    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4f352f-5591-4f2d-bdf7-129518a5f9cc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a4f352f-5591-4f2d-bdf7-129518a5f9cc" (UID: "4a4f352f-5591-4f2d-bdf7-129518a5f9cc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:04:29.116302 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.116279    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a4f352f-5591-4f2d-bdf7-129518a5f9cc-kube-api-access-pwtjf" (OuterVolumeSpecName: "kube-api-access-pwtjf") pod "4a4f352f-5591-4f2d-bdf7-129518a5f9cc" (UID: "4a4f352f-5591-4f2d-bdf7-129518a5f9cc"). InnerVolumeSpecName "kube-api-access-pwtjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:04:29.214951 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.214855    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4f352f-5591-4f2d-bdf7-129518a5f9cc-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:29.214951 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.214892    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwtjf\" (UniqueName: \"kubernetes.io/projected/4a4f352f-5591-4f2d-bdf7-129518a5f9cc-kube-api-access-pwtjf\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:04:29.256923 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.256889    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:29.354288 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.354252    2578 generic.go:356] "Generic (PLEG): container finished" podID="4a4f352f-5591-4f2d-bdf7-129518a5f9cc" containerID="399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f" exitCode=0
Sep 04 23:04:29.354462 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.354298    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"
Sep 04 23:04:29.354462 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.354312    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr" event={"ID":"4a4f352f-5591-4f2d-bdf7-129518a5f9cc","Type":"ContainerDied","Data":"399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f"}
Sep 04 23:04:29.354462 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.354353    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr" event={"ID":"4a4f352f-5591-4f2d-bdf7-129518a5f9cc","Type":"ContainerDied","Data":"17f84d955666de307b95930fb478580413dd7252865bbf1d8fee57c8a4f6823c"}
Sep 04 23:04:29.354462 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.354376    2578 scope.go:117] "RemoveContainer" containerID="399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f"
Sep 04 23:04:29.370222 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.370200    2578 scope.go:117] "RemoveContainer" containerID="399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f"
Sep 04 23:04:29.370555 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:04:29.370529    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f\": container with ID starting with 399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f not found: ID does not exist" containerID="399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f"
Sep 04 23:04:29.370628 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.370568    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f"} err="failed to get container status \"399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f\": rpc error: code = NotFound desc = could not find container \"399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f\": container with ID starting with 399b0e250e2bac1ff8c6705daf6b2cbec906f26a791518cd0b3998818ec55e4f not found: ID does not exist"
Sep 04 23:04:29.391401 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.391374    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"]
Sep 04 23:04:29.399990 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:29.399959    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4fd6f6-749d7b778c-vgqcr"]
Sep 04 23:04:30.389774 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:30.389742    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:30.419381 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:30.419344    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a4f352f-5591-4f2d-bdf7-129518a5f9cc" path="/var/lib/kubelet/pods/4a4f352f-5591-4f2d-bdf7-129518a5f9cc/volumes"
Sep 04 23:04:30.419747 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:30.419733    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fed10899-9e21-44c7-8440-40a7e96b07a0" path="/var/lib/kubelet/pods/fed10899-9e21-44c7-8440-40a7e96b07a0/volumes"
Sep 04 23:04:30.455955 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:30.455914    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:30.554697 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:30.554663    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:30.654767 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:30.654674    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:30.757025 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:30.756984    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:31.361747 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:31.361700    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:31.454879 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:31.454836    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:31.555096 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:31.555064    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:33.127431 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:33.127385    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:35.024549 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:35.024475    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:35.099338 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:35.099311    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:35.292004 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:35.291914    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:35.863543 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:35.863500    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:36.135685 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.135579    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"]
Sep 04 23:04:36.136205 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.136182    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerName="istio-proxy"
Sep 04 23:04:36.136296 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.136211    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerName="istio-proxy"
Sep 04 23:04:36.136296 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.136223    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerName="istio-proxy"
Sep 04 23:04:36.136296 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.136235    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a9edb7b-3f1b-428c-af30-e09421930b19" containerName="istio-proxy"
Sep 04 23:04:36.136296 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.136270    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a4f352f-5591-4f2d-bdf7-129518a5f9cc" containerName="payload-processing"
Sep 04 23:04:36.136296 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.136280    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a4f352f-5591-4f2d-bdf7-129518a5f9cc" containerName="payload-processing"
Sep 04 23:04:36.136296 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.136289    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed10899-9e21-44c7-8440-40a7e96b07a0" containerName="revoke-keys"
Sep 04 23:04:36.136296 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.136299    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed10899-9e21-44c7-8440-40a7e96b07a0" containerName="revoke-keys"
Sep 04 23:04:36.136591 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.136387    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed10899-9e21-44c7-8440-40a7e96b07a0" containerName="revoke-keys"
Sep 04 23:04:36.136591 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.136401    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6974e567-2f6c-468a-9a35-64fc79e91e61" containerName="istio-proxy"
Sep 04 23:04:36.136591 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.136418    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a4f352f-5591-4f2d-bdf7-129518a5f9cc" containerName="payload-processing"
Sep 04 23:04:36.140727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.140703    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.143363 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.143336    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-9ec1e5-openshift-default-dockercfg-m5zkq\""
Sep 04 23:04:36.154262 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.154235    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"]
Sep 04 23:04:36.155790 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.155761    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:36.282326 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.282287    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cdww\" (UniqueName: \"kubernetes.io/projected/9a6102b2-41e7-4846-a636-015af1be4912-kube-api-access-2cdww\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.282507 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.282334    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a6102b2-41e7-4846-a636-015af1be4912-istio-token\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.282507 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.282380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-workload-certs\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.282507 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.282428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a6102b2-41e7-4846-a636-015af1be4912-istiod-ca-cert\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.282507 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.282495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-workload-socket\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.282772 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.282582    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-istio-data\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.282772 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.282632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-credential-socket\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.282772 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.282686    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-istio-envoy\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.282772 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.282760    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a6102b2-41e7-4846-a636-015af1be4912-istio-podinfo\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.378563 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.378529    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:36.383837 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.383815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a6102b2-41e7-4846-a636-015af1be4912-istio-podinfo\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.383922 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.383852    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cdww\" (UniqueName: \"kubernetes.io/projected/9a6102b2-41e7-4846-a636-015af1be4912-kube-api-access-2cdww\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.383922 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.383872    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a6102b2-41e7-4846-a636-015af1be4912-istio-token\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.383922 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.383898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-workload-certs\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.383922 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.383914    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a6102b2-41e7-4846-a636-015af1be4912-istiod-ca-cert\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.384130 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.383935    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-workload-socket\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.384130 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.383991    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-istio-data\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.384130 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.384029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-credential-socket\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.384130 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.384056    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-istio-envoy\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.384403 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.384380    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-workload-certs\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.384471 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.384380    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-istio-data\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.384471 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.384443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-workload-socket\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.384570 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.384463    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-credential-socket\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.384941 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.384916    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a6102b2-41e7-4846-a636-015af1be4912-istiod-ca-cert\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.386463 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.386397    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-istio-envoy\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.386550 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.386506    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a6102b2-41e7-4846-a636-015af1be4912-istio-podinfo\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.393563 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.393540    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a6102b2-41e7-4846-a636-015af1be4912-istio-token\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.393906 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.393886    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cdww\" (UniqueName: \"kubernetes.io/projected/9a6102b2-41e7-4846-a636-015af1be4912-kube-api-access-2cdww\") pod \"e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") " pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.451687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.451630    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:36.732751 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.732724    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"]
Sep 04 23:04:36.738103 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:04:36.736069    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a6102b2_41e7_4846_a636_015af1be4912.slice/crio-2805606dd3e79e7a02fb2f23e9559146913c1f623ab90e567636bab9fdb30668 WatchSource:0}: Error finding container 2805606dd3e79e7a02fb2f23e9559146913c1f623ab90e567636bab9fdb30668: Status 404 returned error can't find the container with id 2805606dd3e79e7a02fb2f23e9559146913c1f623ab90e567636bab9fdb30668
Sep 04 23:04:36.739978 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.739938    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:04:36.740075 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.740012    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:04:36.740075 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:36.740040    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:04:37.405652 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:37.405595    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" event={"ID":"9a6102b2-41e7-4846-a636-015af1be4912","Type":"ContainerStarted","Data":"ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b"}
Sep 04 23:04:37.406071 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:37.405673    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" event={"ID":"9a6102b2-41e7-4846-a636-015af1be4912","Type":"ContainerStarted","Data":"2805606dd3e79e7a02fb2f23e9559146913c1f623ab90e567636bab9fdb30668"}
Sep 04 23:04:37.427444 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:37.427393    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" podStartSLOduration=1.4273764930000001 podStartE2EDuration="1.427376493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:04:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:04:37.424098575 +0000 UTC m=+1165.534219177" watchObservedRunningTime="2026-09-04 23:04:37.427376493 +0000 UTC m=+1165.537497079"
Sep 04 23:04:37.452006 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:37.451976    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:37.457158 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:37.457132    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:38.412413 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:38.412372    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:38.424448 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:38.424418    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:38.994710 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:38.994669    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:39.103766 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:39.103734    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:39.322104 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:39.321996    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:39.413077 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:39.413039    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:39.714802 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:39.714769    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:40.468806 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:40.468773    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:40.542153 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:40.542121    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:40.609196 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:40.609163    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:40.962304 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:40.962272    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:41.049804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:41.049771    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:45.712743 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:45.712710    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:45.812287 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:45.812251    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:45.968907 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:45.968807    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:46.264747 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.260269    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"]
Sep 04 23:04:46.275801 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.275763    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"]
Sep 04 23:04:46.276032 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.275899    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.276688 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.275916    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:46.278650 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.278616    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f486484c-openshift-default-dockercfg-nhltr\""
Sep 04 23:04:46.377424 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.377383    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-envoy\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.377596 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.377437    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-credential-socket\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.377596 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.377556    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-workload-socket\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.377596 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.377589    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-podinfo\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.377775 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.377608    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istiod-ca-cert\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.377775 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.377688    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkwzg\" (UniqueName: \"kubernetes.io/projected/7947271e-2e8a-4fbb-9165-6a56d6c24f48-kube-api-access-hkwzg\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.377775 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.377761    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-workload-certs\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.377874 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.377808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-data\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.377874 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.377835    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-token\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.380518 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.380478    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:46.478307 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478268    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-workload-socket\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.478307 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478304    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-podinfo\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.478569 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478324    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istiod-ca-cert\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.478569 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478342    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkwzg\" (UniqueName: \"kubernetes.io/projected/7947271e-2e8a-4fbb-9165-6a56d6c24f48-kube-api-access-hkwzg\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.478569 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478392    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-workload-certs\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.478569 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478407    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-data\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.478569 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478429    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-token\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.478569 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478453    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-envoy\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.478569 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478503    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-credential-socket\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.478980 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478771    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-workload-socket\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.478980 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478789    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-workload-certs\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.478980 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478828    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-data\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.478980 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.478874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-credential-socket\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.479448 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.479427    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istiod-ca-cert\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.480827 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.480807    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-envoy\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.481128 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.481108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-podinfo\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.486730 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.486704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-token\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.486969 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.486947    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkwzg\" (UniqueName: \"kubernetes.io/projected/7947271e-2e8a-4fbb-9165-6a56d6c24f48-kube-api-access-hkwzg\") pod \"e2e-mt-f486484c-openshift-default-7669d8648-sjc52\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") " pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.588084 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.588048    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:46.911557 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.911471    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"]
Sep 04 23:04:46.916862 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:04:46.916834    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7947271e_2e8a_4fbb_9165_6a56d6c24f48.slice/crio-16d2d9c4c95888225e041458bd53310d22b722f2e35aebf2e3d3b19797767d2f WatchSource:0}: Error finding container 16d2d9c4c95888225e041458bd53310d22b722f2e35aebf2e3d3b19797767d2f: Status 404 returned error can't find the container with id 16d2d9c4c95888225e041458bd53310d22b722f2e35aebf2e3d3b19797767d2f
Sep 04 23:04:46.918984 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.918950    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:04:46.919072 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.919015    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:04:46.919072 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:46.919044    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:04:47.472573 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:47.472528    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" event={"ID":"7947271e-2e8a-4fbb-9165-6a56d6c24f48","Type":"ContainerStarted","Data":"3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d"}
Sep 04 23:04:47.472794 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:47.472581    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" event={"ID":"7947271e-2e8a-4fbb-9165-6a56d6c24f48","Type":"ContainerStarted","Data":"16d2d9c4c95888225e041458bd53310d22b722f2e35aebf2e3d3b19797767d2f"}
Sep 04 23:04:47.493840 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:47.493797    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" podStartSLOduration=1.493783631 podStartE2EDuration="1.493783631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:04:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:04:47.491085284 +0000 UTC m=+1175.601205860" watchObservedRunningTime="2026-09-04 23:04:47.493783631 +0000 UTC m=+1175.603904219"
Sep 04 23:04:47.589072 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:47.589031    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:47.594562 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:47.594534    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:48.479602 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:48.479570    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:48.480603 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:48.480582    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:04:49.428390 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:49.428333    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:49.520324 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:49.520271    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:50.572447 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:50.572400    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:51.874536 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:51.874490    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:04:53.950759 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:53.950720    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": read tcp 10.133.0.2:55874->10.133.0.70:15021: read: connection reset by peer" start-of-body=
Sep 04 23:04:53.951131 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:53.950815    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": read tcp 10.133.0.2:55874->10.133.0.70:15021: read: connection reset by peer"
Sep 04 23:04:54.519135 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:54.519100    2578 generic.go:356] "Generic (PLEG): container finished" podID="9a6102b2-41e7-4846-a636-015af1be4912" containerID="ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b" exitCode=0
Sep 04 23:04:54.519333 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:54.519159    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" event={"ID":"9a6102b2-41e7-4846-a636-015af1be4912","Type":"ContainerDied","Data":"ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b"}
Sep 04 23:04:54.519616 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:54.519597    2578 scope.go:117] "RemoveContainer" containerID="ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b"
Sep 04 23:04:54.520831 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:54.520798    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:04:54.520921 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:54.520875    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:04:54.520921 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:54.520909    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:04:55.527100 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:55.527069    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" event={"ID":"9a6102b2-41e7-4846-a636-015af1be4912","Type":"ContainerStarted","Data":"c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1"}
Sep 04 23:04:56.452577 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:56.452525    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:04:56.453010 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:56.452979    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 04 23:04:56.453212 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:56.453182    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 04 23:04:57.452343 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:57.452303    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 04 23:04:57.452933 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:57.452897    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 04 23:04:58.452845 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:58.452804    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 04 23:04:58.453353 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:58.452885    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 04 23:04:59.452695 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:59.452634    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 04 23:04:59.453043 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:04:59.453012    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 04 23:05:00.452368 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:00.452330    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 04 23:05:00.452733 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:00.452692    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 04 23:05:01.379630 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:01.379595    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:01.458054 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:01.458017    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:05:01.458475 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:01.458446    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:05:01.462061 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:01.462041    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:05:04.286365 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.286319    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:04.612043 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.612011    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:04.892569 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.892485    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"]
Sep 04 23:05:04.895651 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.895612    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:04.898607 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.898422    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-146c0e-openshift-default-dockercfg-qj7xc\""
Sep 04 23:05:04.909578 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.909536    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"]
Sep 04 23:05:04.918306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.918276    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:04.961920 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.961877    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-workload-socket\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:04.962146 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.961925    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5425fba2-c853-480c-b386-1ae2b4dbfc40-istiod-ca-cert\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:04.962146 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.961960    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fggwb\" (UniqueName: \"kubernetes.io/projected/5425fba2-c853-480c-b386-1ae2b4dbfc40-kube-api-access-fggwb\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:04.962146 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.962012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-credential-socket\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:04.962146 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.962075    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-workload-certs\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:04.962146 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.962114    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-envoy\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:04.962146 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.962144    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-podinfo\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:04.962496 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.962181    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-token\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:04.962496 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:04.962209    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-data\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.063527 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.063467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-workload-socket\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.063527 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.063525    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5425fba2-c853-480c-b386-1ae2b4dbfc40-istiod-ca-cert\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.063804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.063559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fggwb\" (UniqueName: \"kubernetes.io/projected/5425fba2-c853-480c-b386-1ae2b4dbfc40-kube-api-access-fggwb\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.063804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.063601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-credential-socket\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.063804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.063693    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-workload-certs\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.063804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.063733    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-envoy\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.063804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.063764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-podinfo\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.064027 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.063806    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-token\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.064027 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.063837    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-data\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.064751 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.064312    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-data\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.064751 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.064430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-credential-socket\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.064751 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.064660    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-workload-socket\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.065004 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.064864    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-workload-certs\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.066054 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.066008    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5425fba2-c853-480c-b386-1ae2b4dbfc40-istiod-ca-cert\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.067701 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.067678    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-envoy\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.067843 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.067827    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-podinfo\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.073020 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.072996    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-token\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.073178 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.073159    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fggwb\" (UniqueName: \"kubernetes.io/projected/5425fba2-c853-480c-b386-1ae2b4dbfc40-kube-api-access-fggwb\") pod \"e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") " pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.087667 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.087612    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:05.212113 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.212021    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:05.500585 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.500557    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"]
Sep 04 23:05:05.502486 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:05:05.502448    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5425fba2_c853_480c_b386_1ae2b4dbfc40.slice/crio-17924ffa1c7955e8d7320dfb48ac80f189fedcd756ac8afbcfed99f98ea248ce WatchSource:0}: Error finding container 17924ffa1c7955e8d7320dfb48ac80f189fedcd756ac8afbcfed99f98ea248ce: Status 404 returned error can't find the container with id 17924ffa1c7955e8d7320dfb48ac80f189fedcd756ac8afbcfed99f98ea248ce
Sep 04 23:05:05.504998 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.504962    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:05:05.505111 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.505052    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:05:05.505111 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.505098    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:05:05.610792 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.610680    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" event={"ID":"5425fba2-c853-480c-b386-1ae2b4dbfc40","Type":"ContainerStarted","Data":"f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340"}
Sep 04 23:05:05.610792 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.610746    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" event={"ID":"5425fba2-c853-480c-b386-1ae2b4dbfc40","Type":"ContainerStarted","Data":"17924ffa1c7955e8d7320dfb48ac80f189fedcd756ac8afbcfed99f98ea248ce"}
Sep 04 23:05:05.637263 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:05.637200    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" podStartSLOduration=1.6371775560000001 podStartE2EDuration="1.637177556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:05:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:05:05.633688887 +0000 UTC m=+1193.743809471" watchObservedRunningTime="2026-09-04 23:05:05.637177556 +0000 UTC m=+1193.747298143"
Sep 04 23:05:06.212265 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:06.212218    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:06.218530 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:06.218500    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:06.618582 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:06.618539    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:06.619580 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:06.619554    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:07.531840 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:07.531802    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:07.622448 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:07.622368    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:07.626576 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:07.626553    2578 generic.go:356] "Generic (PLEG): container finished" podID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerID="3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d" exitCode=0
Sep 04 23:05:07.626727 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:07.626628    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" event={"ID":"7947271e-2e8a-4fbb-9165-6a56d6c24f48","Type":"ContainerDied","Data":"3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d"}
Sep 04 23:05:07.627164 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:07.627146    2578 scope.go:117] "RemoveContainer" containerID="3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d"
Sep 04 23:05:07.628207 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:07.628176    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:05:07.628304 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:07.628246    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:05:07.628304 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:07.628282    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:05:08.638245 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:08.638208    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" event={"ID":"7947271e-2e8a-4fbb-9165-6a56d6c24f48","Type":"ContainerStarted","Data":"0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121"}
Sep 04 23:05:08.684656 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:08.684600    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:08.783012 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:08.782976    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:08.990827 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:08.990738    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:09.291799 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:09.291721    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:09.390921 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:09.390889    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:09.589090 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:09.588994    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:05:09.589331 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:09.589297    2578 patch_prober.go:28] interesting pod/e2e-mt-f486484c-openshift-default-7669d8648-sjc52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 04 23:05:09.589401 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:09.589366    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 04 23:05:10.588791 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:10.588754    2578 patch_prober.go:28] interesting pod/e2e-mt-f486484c-openshift-default-7669d8648-sjc52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 04 23:05:10.589190 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:10.588811    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 04 23:05:11.463028 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:11.462983    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:11.589031 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:11.588993    2578 patch_prober.go:28] interesting pod/e2e-mt-f486484c-openshift-default-7669d8648-sjc52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 04 23:05:11.589406 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:11.589050    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 04 23:05:12.459664 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:12.456179    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 23:05:12.465343 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:12.465319    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 23:05:12.589282 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:12.589248    2578 patch_prober.go:28] interesting pod/e2e-mt-f486484c-openshift-default-7669d8648-sjc52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 04 23:05:12.589683 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:12.589312    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 04 23:05:13.588852 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:13.588812    2578 patch_prober.go:28] interesting pod/e2e-mt-f486484c-openshift-default-7669d8648-sjc52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 04 23:05:13.589035 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:13.588887    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 04 23:05:14.434922 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:14.434888    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:14.592942 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:14.592909    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:05:14.593266 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:14.593241    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:05:14.593910 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:14.593885    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:05:15.071387 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:15.071360    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:15.171287 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:15.171229    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:15.490444 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:15.490407    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:21.419881 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:21.419849    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:21.520328 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:21.520295    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:21.768320 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:21.768238    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p"]
Sep 04 23:05:21.773184 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:21.773167    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p"
Sep 04 23:05:21.775987 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:21.775954    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:05:21.779311 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:21.779286    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p"]
Sep 04 23:05:21.852280 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:21.852223    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/337a0d91-7cae-4216-8121-abc7b3ab38e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f486484c-fdg7p\" (UID: \"337a0d91-7cae-4216-8121-abc7b3ab38e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p"
Sep 04 23:05:21.953333 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:21.953274    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/337a0d91-7cae-4216-8121-abc7b3ab38e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f486484c-fdg7p\" (UID: \"337a0d91-7cae-4216-8121-abc7b3ab38e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p"
Sep 04 23:05:21.954491 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:21.954461    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/337a0d91-7cae-4216-8121-abc7b3ab38e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f486484c-fdg7p\" (UID: \"337a0d91-7cae-4216-8121-abc7b3ab38e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p"
Sep 04 23:05:22.084971 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:22.084940    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p"
Sep 04 23:05:22.378008 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:22.377985    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p"]
Sep 04 23:05:22.380242 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:05:22.380210    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337a0d91_7cae_4216_8121_abc7b3ab38e8.slice/crio-8c4805e294d98a7c66822a0e8b454a15df485ee9051723ef685919237a9e705f WatchSource:0}: Error finding container 8c4805e294d98a7c66822a0e8b454a15df485ee9051723ef685919237a9e705f: Status 404 returned error can't find the container with id 8c4805e294d98a7c66822a0e8b454a15df485ee9051723ef685919237a9e705f
Sep 04 23:05:22.749790 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:22.749745    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p" event={"ID":"337a0d91-7cae-4216-8121-abc7b3ab38e8","Type":"ContainerStarted","Data":"54b6bbb7e4d45f27bf06ecb3572d79971fa083080feb38b9db5d7fe3c20a4fd7"}
Sep 04 23:05:22.750286 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:22.749818    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p" event={"ID":"337a0d91-7cae-4216-8121-abc7b3ab38e8","Type":"ContainerStarted","Data":"8c4805e294d98a7c66822a0e8b454a15df485ee9051723ef685919237a9e705f"}
Sep 04 23:05:22.766375 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:22.766319    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p" podStartSLOduration=1.766301668 podStartE2EDuration="1.766301668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:05:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:05:22.765132266 +0000 UTC m=+1210.875252848" watchObservedRunningTime="2026-09-04 23:05:22.766301668 +0000 UTC m=+1210.876422253"
Sep 04 23:05:23.759216 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:23.759172    2578 generic.go:356] "Generic (PLEG): container finished" podID="337a0d91-7cae-4216-8121-abc7b3ab38e8" containerID="54b6bbb7e4d45f27bf06ecb3572d79971fa083080feb38b9db5d7fe3c20a4fd7" exitCode=0
Sep 04 23:05:23.759627 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:23.759255    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p" event={"ID":"337a0d91-7cae-4216-8121-abc7b3ab38e8","Type":"ContainerDied","Data":"54b6bbb7e4d45f27bf06ecb3572d79971fa083080feb38b9db5d7fe3c20a4fd7"}
Sep 04 23:05:24.911045 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:24.911019    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p"
Sep 04 23:05:25.094669 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:25.094623    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/337a0d91-7cae-4216-8121-abc7b3ab38e8-maas-api-service-ca\") pod \"337a0d91-7cae-4216-8121-abc7b3ab38e8\" (UID: \"337a0d91-7cae-4216-8121-abc7b3ab38e8\") "
Sep 04 23:05:25.095110 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:25.095080    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/337a0d91-7cae-4216-8121-abc7b3ab38e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "337a0d91-7cae-4216-8121-abc7b3ab38e8" (UID: "337a0d91-7cae-4216-8121-abc7b3ab38e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:05:25.196012 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:25.195971    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/337a0d91-7cae-4216-8121-abc7b3ab38e8-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:05:25.774094 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:25.774059    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p"
Sep 04 23:05:25.774267 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:25.774091    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p" event={"ID":"337a0d91-7cae-4216-8121-abc7b3ab38e8","Type":"ContainerDied","Data":"8c4805e294d98a7c66822a0e8b454a15df485ee9051723ef685919237a9e705f"}
Sep 04 23:05:25.774267 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:25.774141    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c4805e294d98a7c66822a0e8b454a15df485ee9051723ef685919237a9e705f"
Sep 04 23:05:26.112407 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:05:26.112376    2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@6-19445-0.service\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5425fba2_c853_480c_b386_1ae2b4dbfc40.slice/crio-f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340.scope\": RecentStats: unable to find data in memory cache]"
Sep 04 23:05:26.781843 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:26.781810    2578 generic.go:356] "Generic (PLEG): container finished" podID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerID="f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340" exitCode=0
Sep 04 23:05:26.782032 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:26.781884    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" event={"ID":"5425fba2-c853-480c-b386-1ae2b4dbfc40","Type":"ContainerDied","Data":"f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340"}
Sep 04 23:05:26.782295 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:26.782281    2578 scope.go:117] "RemoveContainer" containerID="f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340"
Sep 04 23:05:26.783322 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:26.783294    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:05:26.783397 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:26.783363    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:05:26.783442 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:26.783398    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:05:27.789466 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:27.789434    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" event={"ID":"5425fba2-c853-480c-b386-1ae2b4dbfc40","Type":"ContainerStarted","Data":"a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb"}
Sep 04 23:05:28.212984 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:28.212950    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:28.213203 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:28.213182    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 04 23:05:28.213262 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:28.213230    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 04 23:05:29.212739 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:29.212698    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 04 23:05:29.213207 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:29.212759    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 04 23:05:30.213309 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:30.213271    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 04 23:05:30.213851 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:30.213349    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 04 23:05:30.752287 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:30.752245    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:31.213252 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:31.213211    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 04 23:05:31.213439 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:31.213274    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 04 23:05:31.876536 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:31.876500    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:31.970042 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:31.970005    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:32.069481 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:32.069440    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:32.131307 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:32.131233    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:32.203443 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:32.203397    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p"]
Sep 04 23:05:32.207841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:32.207815    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f486484c-fdg7p"]
Sep 04 23:05:32.210594 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:32.210568    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:32.212729 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:32.212707    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 04 23:05:32.212807 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:32.212756    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 04 23:05:32.420318 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:32.420225    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="337a0d91-7cae-4216-8121-abc7b3ab38e8" path="/var/lib/kubelet/pods/337a0d91-7cae-4216-8121-abc7b3ab38e8/volumes"
Sep 04 23:05:32.557157 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:32.557121    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:32.656865 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:32.656827    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:33.216519 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:33.216484    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:33.216859 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:33.216838    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:33.217515 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:33.217492    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:05:38.171991 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:38.171956    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:38.272910 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:38.272874    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:38.423008 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:38.422924    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"]
Sep 04 23:05:38.423210 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:38.423185    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerName="istio-proxy" containerID="cri-o://0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121" gracePeriod=30
Sep 04 23:05:38.451534 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:38.451479    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:38.786051 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:38.785967    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:38.950814 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:38.950781    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:39.032978 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:39.032941    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:43.681307 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.681284    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:05:43.790053 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.789976    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-workload-socket\") pod \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") "
Sep 04 23:05:43.790053 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790013    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-credential-socket\") pod \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") "
Sep 04 23:05:43.790053 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790045    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-data\") pod \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") "
Sep 04 23:05:43.790345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790100    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-podinfo\") pod \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") "
Sep 04 23:05:43.790345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790124    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-envoy\") pod \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") "
Sep 04 23:05:43.790345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790182    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-workload-certs\") pod \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") "
Sep 04 23:05:43.790345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790221    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkwzg\" (UniqueName: \"kubernetes.io/projected/7947271e-2e8a-4fbb-9165-6a56d6c24f48-kube-api-access-hkwzg\") pod \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") "
Sep 04 23:05:43.790345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790256    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-token\") pod \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") "
Sep 04 23:05:43.790345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790250    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7947271e-2e8a-4fbb-9165-6a56d6c24f48" (UID: "7947271e-2e8a-4fbb-9165-6a56d6c24f48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:05:43.790345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790275    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7947271e-2e8a-4fbb-9165-6a56d6c24f48" (UID: "7947271e-2e8a-4fbb-9165-6a56d6c24f48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:05:43.790345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790302    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istiod-ca-cert\") pod \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\" (UID: \"7947271e-2e8a-4fbb-9165-6a56d6c24f48\") "
Sep 04 23:05:43.790815 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790385    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-data" (OuterVolumeSpecName: "istio-data") pod "7947271e-2e8a-4fbb-9165-6a56d6c24f48" (UID: "7947271e-2e8a-4fbb-9165-6a56d6c24f48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:05:43.790815 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790537    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7947271e-2e8a-4fbb-9165-6a56d6c24f48" (UID: "7947271e-2e8a-4fbb-9165-6a56d6c24f48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:05:43.790815 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790632    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:05:43.790815 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790678    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:05:43.790815 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790692    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:05:43.790815 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.790732    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7947271e-2e8a-4fbb-9165-6a56d6c24f48" (UID: "7947271e-2e8a-4fbb-9165-6a56d6c24f48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:05:43.792386 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.792362    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7947271e-2e8a-4fbb-9165-6a56d6c24f48-kube-api-access-hkwzg" (OuterVolumeSpecName: "kube-api-access-hkwzg") pod "7947271e-2e8a-4fbb-9165-6a56d6c24f48" (UID: "7947271e-2e8a-4fbb-9165-6a56d6c24f48"). InnerVolumeSpecName "kube-api-access-hkwzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:05:43.792491 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.792458    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-token" (OuterVolumeSpecName: "istio-token") pod "7947271e-2e8a-4fbb-9165-6a56d6c24f48" (UID: "7947271e-2e8a-4fbb-9165-6a56d6c24f48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:05:43.792778 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.792760    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7947271e-2e8a-4fbb-9165-6a56d6c24f48" (UID: "7947271e-2e8a-4fbb-9165-6a56d6c24f48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:05:43.792778 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.792765    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7947271e-2e8a-4fbb-9165-6a56d6c24f48" (UID: "7947271e-2e8a-4fbb-9165-6a56d6c24f48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:05:43.888475 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.888442    2578 generic.go:356] "Generic (PLEG): container finished" podID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerID="0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121" exitCode=0
Sep 04 23:05:43.888613 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.888502    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"
Sep 04 23:05:43.888613 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.888517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" event={"ID":"7947271e-2e8a-4fbb-9165-6a56d6c24f48","Type":"ContainerDied","Data":"0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121"}
Sep 04 23:05:43.888613 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.888593    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52" event={"ID":"7947271e-2e8a-4fbb-9165-6a56d6c24f48","Type":"ContainerDied","Data":"16d2d9c4c95888225e041458bd53310d22b722f2e35aebf2e3d3b19797767d2f"}
Sep 04 23:05:43.888758 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.888615    2578 scope.go:117] "RemoveContainer" containerID="0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121"
Sep 04 23:05:43.891665 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.891630    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:05:43.891758 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.891671    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:05:43.891758 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.891686    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7947271e-2e8a-4fbb-9165-6a56d6c24f48-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:05:43.891758 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.891701    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkwzg\" (UniqueName: \"kubernetes.io/projected/7947271e-2e8a-4fbb-9165-6a56d6c24f48-kube-api-access-hkwzg\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:05:43.891758 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.891715    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:05:43.891758 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.891730    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7947271e-2e8a-4fbb-9165-6a56d6c24f48-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:05:43.898084 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.898068    2578 scope.go:117] "RemoveContainer" containerID="3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d"
Sep 04 23:05:43.906153 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.906135    2578 scope.go:117] "RemoveContainer" containerID="0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121"
Sep 04 23:05:43.906405 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:05:43.906386    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121\": container with ID starting with 0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121 not found: ID does not exist" containerID="0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121"
Sep 04 23:05:43.906460 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.906413    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121"} err="failed to get container status \"0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121\": rpc error: code = NotFound desc = could not find container \"0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121\": container with ID starting with 0fd99e991993820a6f0f59b48be1b086b6c99612dfe220c90b6b84b2f251f121 not found: ID does not exist"
Sep 04 23:05:43.906460 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.906430    2578 scope.go:117] "RemoveContainer" containerID="3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d"
Sep 04 23:05:43.906666 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:05:43.906632    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d\": container with ID starting with 3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d not found: ID does not exist" containerID="3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d"
Sep 04 23:05:43.906745 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.906672    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d"} err="failed to get container status \"3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d\": rpc error: code = NotFound desc = could not find container \"3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d\": container with ID starting with 3ac151483cd234b05a9e99428497579a6b2d68cdf60f4c9b78b96b52c742606d not found: ID does not exist"
Sep 04 23:05:43.912151 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.912128    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"]
Sep 04 23:05:43.918112 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:43.918078    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f486484c-openshift-default-7669d8648-sjc52"]
Sep 04 23:05:44.420165 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:44.420134    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" path="/var/lib/kubelet/pods/7947271e-2e8a-4fbb-9165-6a56d6c24f48/volumes"
Sep 04 23:05:44.732278 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:44.732197    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:44.791206 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:44.791172    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:44.858527 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:44.858486    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:44.957816 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:44.957783    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:45.089078 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:45.089043    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:45.723515 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:45.723485    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:51.123126 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:51.123090    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:51.190571 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:51.190538    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:52.042595 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.042557    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:52.353820 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.353783    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"]
Sep 04 23:05:52.354306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.354203    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="337a0d91-7cae-4216-8121-abc7b3ab38e8" containerName="revoke-keys"
Sep 04 23:05:52.354306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.354216    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="337a0d91-7cae-4216-8121-abc7b3ab38e8" containerName="revoke-keys"
Sep 04 23:05:52.354306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.354232    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerName="istio-proxy"
Sep 04 23:05:52.354306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.354238    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerName="istio-proxy"
Sep 04 23:05:52.354306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.354253    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerName="istio-proxy"
Sep 04 23:05:52.354306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.354259    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerName="istio-proxy"
Sep 04 23:05:52.354677 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.354314    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerName="istio-proxy"
Sep 04 23:05:52.354677 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.354322    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7947271e-2e8a-4fbb-9165-6a56d6c24f48" containerName="istio-proxy"
Sep 04 23:05:52.354677 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.354331    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="337a0d91-7cae-4216-8121-abc7b3ab38e8" containerName="revoke-keys"
Sep 04 23:05:52.359333 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.359311    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.361944 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.361923    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-ed43d3-openshift-default-dockercfg-zmj5z\""
Sep 04 23:05:52.371395 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.371368    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"]
Sep 04 23:05:52.383827 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.383801    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:52.479601 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.479558    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-istio-envoy\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.479803 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.479620    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/342cede2-d66e-4584-a747-ba5abbf72b10-istiod-ca-cert\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.479803 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.479705    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-istio-data\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.479803 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.479737    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-workload-socket\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.479803 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.479774    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-workload-certs\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.479803 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.479791    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/342cede2-d66e-4584-a747-ba5abbf72b10-istio-podinfo\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.480008 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.479813    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zj7k\" (UniqueName: \"kubernetes.io/projected/342cede2-d66e-4584-a747-ba5abbf72b10-kube-api-access-6zj7k\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.480008 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.479860    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/342cede2-d66e-4584-a747-ba5abbf72b10-istio-token\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.480008 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.479907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-credential-socket\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581282 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-workload-certs\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581282 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/342cede2-d66e-4584-a747-ba5abbf72b10-istio-podinfo\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581551 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581303    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zj7k\" (UniqueName: \"kubernetes.io/projected/342cede2-d66e-4584-a747-ba5abbf72b10-kube-api-access-6zj7k\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581551 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581331    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/342cede2-d66e-4584-a747-ba5abbf72b10-istio-token\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581551 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581368    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-credential-socket\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581551 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-istio-envoy\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581551 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581474    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/342cede2-d66e-4584-a747-ba5abbf72b10-istiod-ca-cert\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581551 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581510    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-istio-data\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581551 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581534    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-workload-socket\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581975 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-workload-certs\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581975 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581830    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-credential-socket\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581975 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581935    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-istio-data\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.581975 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.581971    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-workload-socket\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.582514 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.582489    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/342cede2-d66e-4584-a747-ba5abbf72b10-istiod-ca-cert\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.583867 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.583847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-istio-envoy\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.584151 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.584132    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/342cede2-d66e-4584-a747-ba5abbf72b10-istio-podinfo\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.591473 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.591441    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zj7k\" (UniqueName: \"kubernetes.io/projected/342cede2-d66e-4584-a747-ba5abbf72b10-kube-api-access-6zj7k\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.591587 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.591560    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/342cede2-d66e-4584-a747-ba5abbf72b10-istio-token\") pod \"e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") " pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.670959 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.670868    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:52.731217 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.731159    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:52.953317 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.953289    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"]
Sep 04 23:05:52.954338 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:05:52.954314    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod342cede2_d66e_4584_a747_ba5abbf72b10.slice/crio-fc512ea5a172e8d91454d24f632b8adb84fdfe197cd01b472d2d2b771434350f WatchSource:0}: Error finding container fc512ea5a172e8d91454d24f632b8adb84fdfe197cd01b472d2d2b771434350f: Status 404 returned error can't find the container with id fc512ea5a172e8d91454d24f632b8adb84fdfe197cd01b472d2d2b771434350f
Sep 04 23:05:52.956473 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.956438    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:05:52.956559 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.956513    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:05:52.956559 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:52.956546    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:05:53.965482 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:53.965446    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" event={"ID":"342cede2-d66e-4584-a747-ba5abbf72b10","Type":"ContainerStarted","Data":"435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb"}
Sep 04 23:05:53.965904 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:53.965502    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" event={"ID":"342cede2-d66e-4584-a747-ba5abbf72b10","Type":"ContainerStarted","Data":"fc512ea5a172e8d91454d24f632b8adb84fdfe197cd01b472d2d2b771434350f"}
Sep 04 23:05:53.989250 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:53.989197    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" podStartSLOduration=1.989180905 podStartE2EDuration="1.989180905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:05:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:05:53.986346634 +0000 UTC m=+1242.096467225" watchObservedRunningTime="2026-09-04 23:05:53.989180905 +0000 UTC m=+1242.099301490"
Sep 04 23:05:54.671835 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:54.671793    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:54.677256 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:54.677225    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:54.973121 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:54.973023    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:54.973981 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:54.973961    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:05:55.925389 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:55.925355    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:56.021434 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:56.021404    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:57.071794 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.071749    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:57.172072 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.172032    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:57.486361 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.486327    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:57.590905 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.590859    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:57.682347 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.682308    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:57.687175 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.687148    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"]
Sep 04 23:05:57.691900 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.691878    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"
Sep 04 23:05:57.694635 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.694615    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-ed43d3\""
Sep 04 23:05:57.694794 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.694777    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-ed43d3-dockercfg-x5vcx\""
Sep 04 23:05:57.702192 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.702165    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"]
Sep 04 23:05:57.833698 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.833661    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkt5f\" (UniqueName: \"kubernetes.io/projected/855ef520-6260-427b-abea-412e97ea30ec-kube-api-access-qkt5f\") pod \"payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb\" (UID: \"855ef520-6260-427b-abea-412e97ea30ec\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"
Sep 04 23:05:57.833698 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.833700    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855ef520-6260-427b-abea-412e97ea30ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb\" (UID: \"855ef520-6260-427b-abea-412e97ea30ec\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"
Sep 04 23:05:57.934556 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.934516    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkt5f\" (UniqueName: \"kubernetes.io/projected/855ef520-6260-427b-abea-412e97ea30ec-kube-api-access-qkt5f\") pod \"payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb\" (UID: \"855ef520-6260-427b-abea-412e97ea30ec\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"
Sep 04 23:05:57.934766 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.934560    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855ef520-6260-427b-abea-412e97ea30ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb\" (UID: \"855ef520-6260-427b-abea-412e97ea30ec\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"
Sep 04 23:05:57.935330 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.935308    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855ef520-6260-427b-abea-412e97ea30ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb\" (UID: \"855ef520-6260-427b-abea-412e97ea30ec\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"
Sep 04 23:05:57.942072 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:57.942046    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkt5f\" (UniqueName: \"kubernetes.io/projected/855ef520-6260-427b-abea-412e97ea30ec-kube-api-access-qkt5f\") pod \"payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb\" (UID: \"855ef520-6260-427b-abea-412e97ea30ec\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"
Sep 04 23:05:58.004474 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:58.004439    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"
Sep 04 23:05:58.164028 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:58.163928    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:05:58.287222 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:58.287187    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"]
Sep 04 23:05:58.288689 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:05:58.288636    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855ef520_6260_427b_abea_412e97ea30ec.slice/crio-3b6bbf5f12981b4374fce91424bbd6d7a42cb35966075da01f1d74083d81ff5d WatchSource:0}: Error finding container 3b6bbf5f12981b4374fce91424bbd6d7a42cb35966075da01f1d74083d81ff5d: Status 404 returned error can't find the container with id 3b6bbf5f12981b4374fce91424bbd6d7a42cb35966075da01f1d74083d81ff5d
Sep 04 23:05:58.997675 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:58.997617    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb" event={"ID":"855ef520-6260-427b-abea-412e97ea30ec","Type":"ContainerStarted","Data":"6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9"}
Sep 04 23:05:58.997844 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:58.997734    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb" event={"ID":"855ef520-6260-427b-abea-412e97ea30ec","Type":"ContainerStarted","Data":"3b6bbf5f12981b4374fce91424bbd6d7a42cb35966075da01f1d74083d81ff5d"}
Sep 04 23:05:58.997844 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:58.997765    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"
Sep 04 23:05:59.017428 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:05:59.017373    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb" podStartSLOduration=2.017358851 podStartE2EDuration="2.017358851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:05:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:05:59.014332514 +0000 UTC m=+1247.124453099" watchObservedRunningTime="2026-09-04 23:05:59.017358851 +0000 UTC m=+1247.127479438"
Sep 04 23:06:10.006446 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:10.006407    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"
Sep 04 23:06:11.014873 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:11.014814    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 04 23:06:11.015376 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:11.014929    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 04 23:06:13.100838 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:13.100802    2578 generic.go:356] "Generic (PLEG): container finished" podID="342cede2-d66e-4584-a747-ba5abbf72b10" containerID="435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb" exitCode=0
Sep 04 23:06:13.101238 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:13.100875    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" event={"ID":"342cede2-d66e-4584-a747-ba5abbf72b10","Type":"ContainerDied","Data":"435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb"}
Sep 04 23:06:13.101345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:13.101325    2578 scope.go:117] "RemoveContainer" containerID="435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb"
Sep 04 23:06:13.102444 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:13.102406    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:06:13.102576 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:13.102497    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:06:13.102576 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:13.102545    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:06:14.108832 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:14.108794    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" event={"ID":"342cede2-d66e-4584-a747-ba5abbf72b10","Type":"ContainerStarted","Data":"4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0"}
Sep 04 23:06:14.671795 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:14.671746    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:06:14.672151 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:14.672123    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 04 23:06:14.672231 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:14.672188    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 04 23:06:15.672051 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:15.672015    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 04 23:06:15.672470 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:15.672071    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 04 23:06:16.671704 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:16.671669    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 04 23:06:16.671890 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:16.671727    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 04 23:06:17.671634 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:17.671588    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 04 23:06:17.672072 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:17.671694    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 04 23:06:18.671621 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:18.671583    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 04 23:06:18.671850 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:18.671678    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 04 23:06:19.675917 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:19.675885    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:06:19.676303 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:19.676139    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:06:19.676902 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:19.676879    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:06:20.726755 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:20.726724    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:06:20.930543 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:20.930510    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:06:21.290509 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.290456    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:06:21.577156 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.573849    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"]
Sep 04 23:06:21.578531 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.578501    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.581396 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.581368    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-217ad6-openshift-default-dockercfg-b26pk\""
Sep 04 23:06:21.597211 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.597177    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:06:21.598794 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.598765    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"]
Sep 04 23:06:21.658604 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.658558    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-data\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.658833 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.658700    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istiod-ca-cert\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.658833 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.658764    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-workload-certs\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.658833 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.658792    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-workload-socket\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.658833 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.658819    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-credential-socket\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.659053 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.658840    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-podinfo\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.659053 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.658969    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-envoy\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.659164 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.659066    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbw45\" (UniqueName: \"kubernetes.io/projected/1262d309-5b05-42aa-ba2c-5c0f0f703dae-kube-api-access-rbw45\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.659164 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.659152    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-token\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.746623 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.746588    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:06:21.759900 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.759873    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbw45\" (UniqueName: \"kubernetes.io/projected/1262d309-5b05-42aa-ba2c-5c0f0f703dae-kube-api-access-rbw45\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760042 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.759925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-token\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760042 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.759962    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-data\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760042 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.759987    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istiod-ca-cert\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760042 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.760027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-workload-certs\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760272 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.760056    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-workload-socket\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760272 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.760083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-credential-socket\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760272 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.760108    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-podinfo\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760272 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.760142    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-envoy\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760454 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.760430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-data\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760510 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.760475    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-workload-certs\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760612 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.760590    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-workload-socket\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760726 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.760704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-credential-socket\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.760849 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.760831    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istiod-ca-cert\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.762537 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.762517    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-envoy\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.762732 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.762712    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-podinfo\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.767913 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.767887    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbw45\" (UniqueName: \"kubernetes.io/projected/1262d309-5b05-42aa-ba2c-5c0f0f703dae-kube-api-access-rbw45\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.768107 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.768085    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-token\") pod \"e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") " pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:21.899295 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:21.899205    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:22.183556 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:22.183528    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"]
Sep 04 23:06:22.184913 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:06:22.184885    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1262d309_5b05_42aa_ba2c_5c0f0f703dae.slice/crio-ece9a984183249ae0e053469145c507b127d51656c6c01f72e81cdac644aebef WatchSource:0}: Error finding container ece9a984183249ae0e053469145c507b127d51656c6c01f72e81cdac644aebef: Status 404 returned error can't find the container with id ece9a984183249ae0e053469145c507b127d51656c6c01f72e81cdac644aebef
Sep 04 23:06:22.187217 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:22.187183    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:06:22.187307 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:22.187250    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:06:22.187307 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:22.187278    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:06:23.167383 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:23.167347    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" event={"ID":"1262d309-5b05-42aa-ba2c-5c0f0f703dae","Type":"ContainerStarted","Data":"49a0b4d03729ed94686ae5d1b52b7cb5d94938f15b666431e52b112605901a4b"}
Sep 04 23:06:23.167830 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:23.167397    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" event={"ID":"1262d309-5b05-42aa-ba2c-5c0f0f703dae","Type":"ContainerStarted","Data":"ece9a984183249ae0e053469145c507b127d51656c6c01f72e81cdac644aebef"}
Sep 04 23:06:23.189921 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:23.189860    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" podStartSLOduration=2.189845073 podStartE2EDuration="2.189845073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:06:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:06:23.187465271 +0000 UTC m=+1271.297585857" watchObservedRunningTime="2026-09-04 23:06:23.189845073 +0000 UTC m=+1271.299965656"
Sep 04 23:06:23.899916 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:23.899873    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:23.905291 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:23.905262    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:24.173585 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:24.173486    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:24.174550 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:24.174529    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:25.100452 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:25.100420    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:06:25.203275 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:25.203242    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:06:26.247808 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:26.247779    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:06:26.317626 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:26.317589    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:06:26.757581 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:26.757551    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:06:26.859302 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:26.859266    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:06:46.329481 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:46.329447    2578 generic.go:356] "Generic (PLEG): container finished" podID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerID="49a0b4d03729ed94686ae5d1b52b7cb5d94938f15b666431e52b112605901a4b" exitCode=0
Sep 04 23:06:46.330008 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:46.329517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" event={"ID":"1262d309-5b05-42aa-ba2c-5c0f0f703dae","Type":"ContainerDied","Data":"49a0b4d03729ed94686ae5d1b52b7cb5d94938f15b666431e52b112605901a4b"}
Sep 04 23:06:46.330008 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:46.329960    2578 scope.go:117] "RemoveContainer" containerID="49a0b4d03729ed94686ae5d1b52b7cb5d94938f15b666431e52b112605901a4b"
Sep 04 23:06:46.331127 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:46.331098    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:06:46.331196 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:46.331167    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:06:46.331233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:46.331198    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:06:47.342194 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:47.342154    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" event={"ID":"1262d309-5b05-42aa-ba2c-5c0f0f703dae","Type":"ContainerStarted","Data":"09ea264f6ab1f1bc7eb67afd751cd24fa7d17594b966e0ba68ac270ec542f97f"}
Sep 04 23:06:47.899504 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:47.899471    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:47.899833 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:47.899809    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 04 23:06:47.899927 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:47.899863    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 04 23:06:48.900529 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:48.900490    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 04 23:06:48.900931 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:48.900551    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 04 23:06:49.900452 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:49.900412    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 04 23:06:49.900653 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:49.900473    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 04 23:06:50.144458 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:50.144422    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:06:50.900544 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:50.900500    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 04 23:06:50.900747 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:50.900573    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 04 23:06:51.900159 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:51.900124    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 04 23:06:51.900343 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:51.900189    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 04 23:06:52.903715 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:52.903678    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:52.904203 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:52.903944    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:06:52.904580 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:06:52.904561    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:07:03.505919 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.505878    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:07:03.820950 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.820866    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"]
Sep 04 23:07:03.824006 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.823984    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.826593 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.826573    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9f64c8d3-openshift-default-dockercfg-v6m64\""
Sep 04 23:07:03.837409 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.837380    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"]
Sep 04 23:07:03.848472 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.848438    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:07:03.868189 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.868159    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-workload-socket\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.868318 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.868215    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-envoy\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.868318 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.868284    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istiod-ca-cert\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.868408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.868351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-credential-socket\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.868408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.868382    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-podinfo\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.868489 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.868420    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-data\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.868489 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.868441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-token\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.868489 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.868471    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-workload-certs\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.868589 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.868505    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgv4z\" (UniqueName: \"kubernetes.io/projected/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-kube-api-access-zgv4z\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.969874 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.969838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-token\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.969874 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.969886    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-workload-certs\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.970133 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.969927    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgv4z\" (UniqueName: \"kubernetes.io/projected/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-kube-api-access-zgv4z\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.970133 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.969962    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-workload-socket\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.970133 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.970008    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-envoy\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.970133 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.970054    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istiod-ca-cert\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.970133 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.970111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-credential-socket\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.970417 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.970141    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-podinfo\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.970417 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.970188    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-data\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.970417 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.970392    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-workload-certs\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.970588 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.970433    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-workload-socket\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.970753 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.970729    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-credential-socket\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.970841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.970771    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-data\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.970901 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.970869    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istiod-ca-cert\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.972369 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.972348    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-envoy\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.972719 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.972696    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-podinfo\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.977981 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.977962    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-token\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:03.978159 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:03.978137    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgv4z\" (UniqueName: \"kubernetes.io/projected/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-kube-api-access-zgv4z\") pod \"e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") " pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:04.041002 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:04.040967    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:07:04.135866 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:04.135830    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:04.413380 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:04.413353    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"]
Sep 04 23:07:04.414772 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:07:04.414747    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b1e3ff_5c8c_4b49_a3b7_c1c820e6676b.slice/crio-3c54e2c33c773d28502653b3c0dce8264e1f66f4a777bc058fc00b971ce6717c WatchSource:0}: Error finding container 3c54e2c33c773d28502653b3c0dce8264e1f66f4a777bc058fc00b971ce6717c: Status 404 returned error can't find the container with id 3c54e2c33c773d28502653b3c0dce8264e1f66f4a777bc058fc00b971ce6717c
Sep 04 23:07:04.416971 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:04.416926    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:07:04.417084 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:04.416988    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:07:04.417084 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:04.417015    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:07:04.466897 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:04.466868    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" event={"ID":"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b","Type":"ContainerStarted","Data":"3c54e2c33c773d28502653b3c0dce8264e1f66f4a777bc058fc00b971ce6717c"}
Sep 04 23:07:05.475349 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:05.475297    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" event={"ID":"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b","Type":"ContainerStarted","Data":"50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892"}
Sep 04 23:07:05.494744 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:05.494699    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" podStartSLOduration=2.494683908 podStartE2EDuration="2.494683908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:07:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:07:05.493004401 +0000 UTC m=+1313.603124989" watchObservedRunningTime="2026-09-04 23:07:05.494683908 +0000 UTC m=+1313.604804493"
Sep 04 23:07:06.136724 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:06.136688    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:06.143389 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:06.143361    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:06.486332 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:06.486239    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:06.487237 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:06.487214    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:07.462919 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:07.462885    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:07:07.555501 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:07.555467    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:07:08.739408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:08.739378    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:07:08.839202 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:08.839166    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:07:08.942230 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:08.942199    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:07:09.230547 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.230516    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:07:09.323789 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.323758    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:07:09.366010 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.365975    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"]
Sep 04 23:07:09.368628 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.368603    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"
Sep 04 23:07:09.370970 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.370948    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9f64c8d3\""
Sep 04 23:07:09.370970 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.370963    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9f64c8d3-dockercfg-nq68c\""
Sep 04 23:07:09.380623 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.380596    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"]
Sep 04 23:07:09.427288 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.427243    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8-plugins-config-volume\") pod \"payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7\" (UID: \"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8\") " pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"
Sep 04 23:07:09.427566 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.427538    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv4jn\" (UniqueName: \"kubernetes.io/projected/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8-kube-api-access-zv4jn\") pod \"payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7\" (UID: \"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8\") " pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"
Sep 04 23:07:09.528587 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.528501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8-plugins-config-volume\") pod \"payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7\" (UID: \"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8\") " pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"
Sep 04 23:07:09.528587 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.528537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv4jn\" (UniqueName: \"kubernetes.io/projected/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8-kube-api-access-zv4jn\") pod \"payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7\" (UID: \"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8\") " pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"
Sep 04 23:07:09.529248 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.529225    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8-plugins-config-volume\") pod \"payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7\" (UID: \"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8\") " pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"
Sep 04 23:07:09.536029 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.535997    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv4jn\" (UniqueName: \"kubernetes.io/projected/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8-kube-api-access-zv4jn\") pod \"payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7\" (UID: \"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8\") " pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"
Sep 04 23:07:09.681822 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.681778    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"
Sep 04 23:07:09.958876 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:09.958841    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"]
Sep 04 23:07:09.960286 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:07:09.960246    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b6d5a43_4bc6_4cb0_879f_77f4ebb7fca8.slice/crio-be9eca0db30b1705e8194c845119ef8792f3d4f1d246f41f5ed5c4832a8d4a26 WatchSource:0}: Error finding container be9eca0db30b1705e8194c845119ef8792f3d4f1d246f41f5ed5c4832a8d4a26: Status 404 returned error can't find the container with id be9eca0db30b1705e8194c845119ef8792f3d4f1d246f41f5ed5c4832a8d4a26
Sep 04 23:07:10.512535 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:10.512495    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7" event={"ID":"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8","Type":"ContainerStarted","Data":"87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098"}
Sep 04 23:07:10.512728 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:10.512562    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"
Sep 04 23:07:10.512728 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:10.512581    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7" event={"ID":"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8","Type":"ContainerStarted","Data":"be9eca0db30b1705e8194c845119ef8792f3d4f1d246f41f5ed5c4832a8d4a26"}
Sep 04 23:07:10.531397 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:10.531347    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7" podStartSLOduration=1.531332156 podStartE2EDuration="1.531332156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:07:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:07:10.529814565 +0000 UTC m=+1318.639935177" watchObservedRunningTime="2026-09-04 23:07:10.531332156 +0000 UTC m=+1318.641452741"
Sep 04 23:07:21.521613 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:21.521575    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"
Sep 04 23:07:31.648134 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:31.648103    2578 generic.go:356] "Generic (PLEG): container finished" podID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerID="50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892" exitCode=0
Sep 04 23:07:31.648558 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:31.648192    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" event={"ID":"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b","Type":"ContainerDied","Data":"50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892"}
Sep 04 23:07:31.648759 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:31.648743    2578 scope.go:117] "RemoveContainer" containerID="50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892"
Sep 04 23:07:31.649946 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:31.649920    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:07:31.650007 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:31.649988    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:07:31.650043 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:31.650022    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:07:32.451007 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:32.450966    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:07:32.662138 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:32.662099    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" event={"ID":"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b","Type":"ContainerStarted","Data":"25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7"}
Sep 04 23:07:33.137131 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:33.137094    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:33.138409 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:33.138374    2578 patch_prober.go:28] interesting pod/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 04 23:07:33.138622 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:33.138592    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 04 23:07:34.137112 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:34.137071    2578 patch_prober.go:28] interesting pod/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 04 23:07:34.137759 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:34.137719    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 04 23:07:35.137401 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:35.137296    2578 patch_prober.go:28] interesting pod/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 04 23:07:35.137401 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:35.137379    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 04 23:07:35.760389 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:35.760354    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:07:36.137210 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:36.137175    2578 patch_prober.go:28] interesting pod/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 04 23:07:36.137486 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:36.137457    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 04 23:07:36.486939 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:36.486856    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:37.136958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:37.136921    2578 patch_prober.go:28] interesting pod/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 04 23:07:37.137276 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:37.137242    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 04 23:07:38.141273 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:38.141242    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:38.142474 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:38.142445    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:07:44.821416 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:07:44.821369    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:08:12.558974 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:12.558943    2578 scope.go:117] "RemoveContainer" containerID="53fb450d0ab90244b26f92afbc0201c2ac126b871828447522f767572a1f9ca6"
Sep 04 23:08:12.568042 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:12.568020    2578 scope.go:117] "RemoveContainer" containerID="91385f296bdb707f960c7180092e95f22e7905f4428ba959d9968e9dbbcaa1a1"
Sep 04 23:08:21.398430 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:21.398394    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:08:21.747259 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:21.747161    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:08:43.549908 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:43.549869    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:08:43.650089 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:43.650049    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:08:43.800958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:43.800865    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:08:47.807325 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:47.807291    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6"]
Sep 04 23:08:47.811203 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:47.811178    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6"
Sep 04 23:08:47.813929 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:47.813905    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:08:47.820904 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:47.820863    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6"]
Sep 04 23:08:47.905859 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:47.905825    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3618a3-4b30-4e83-8ec5-5bb128cf31d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6\" (UID: \"de3618a3-4b30-4e83-8ec5-5bb128cf31d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6"
Sep 04 23:08:47.918532 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:47.918499    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:08:48.007101 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:48.007057    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3618a3-4b30-4e83-8ec5-5bb128cf31d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6\" (UID: \"de3618a3-4b30-4e83-8ec5-5bb128cf31d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6"
Sep 04 23:08:48.007803 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:48.007776    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3618a3-4b30-4e83-8ec5-5bb128cf31d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6\" (UID: \"de3618a3-4b30-4e83-8ec5-5bb128cf31d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6"
Sep 04 23:08:48.130089 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:48.130057    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6"
Sep 04 23:08:48.407606 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:48.407582    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6"]
Sep 04 23:08:48.409148 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:08:48.409123    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde3618a3_4b30_4e83_8ec5_5bb128cf31d4.slice/crio-07982e643a407436bcdfcc336a93a9a6970995484906febc83b66ca6312a42f8 WatchSource:0}: Error finding container 07982e643a407436bcdfcc336a93a9a6970995484906febc83b66ca6312a42f8: Status 404 returned error can't find the container with id 07982e643a407436bcdfcc336a93a9a6970995484906febc83b66ca6312a42f8
Sep 04 23:08:49.173921 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:49.173885    2578 generic.go:356] "Generic (PLEG): container finished" podID="de3618a3-4b30-4e83-8ec5-5bb128cf31d4" containerID="e260abe9dacb187a89550bca5eb25637a8db2e1d63e86982b6d004c2b302be62" exitCode=0
Sep 04 23:08:49.174265 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:49.173963    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6" event={"ID":"de3618a3-4b30-4e83-8ec5-5bb128cf31d4","Type":"ContainerDied","Data":"e260abe9dacb187a89550bca5eb25637a8db2e1d63e86982b6d004c2b302be62"}
Sep 04 23:08:49.174265 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:49.174018    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6" event={"ID":"de3618a3-4b30-4e83-8ec5-5bb128cf31d4","Type":"ContainerStarted","Data":"07982e643a407436bcdfcc336a93a9a6970995484906febc83b66ca6312a42f8"}
Sep 04 23:08:50.315301 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:50.315275    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6"
Sep 04 23:08:50.429390 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:50.429362    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3618a3-4b30-4e83-8ec5-5bb128cf31d4-maas-api-service-ca\") pod \"de3618a3-4b30-4e83-8ec5-5bb128cf31d4\" (UID: \"de3618a3-4b30-4e83-8ec5-5bb128cf31d4\") "
Sep 04 23:08:50.429705 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:50.429684    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de3618a3-4b30-4e83-8ec5-5bb128cf31d4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de3618a3-4b30-4e83-8ec5-5bb128cf31d4" (UID: "de3618a3-4b30-4e83-8ec5-5bb128cf31d4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:08:50.530229 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:50.530143    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3618a3-4b30-4e83-8ec5-5bb128cf31d4-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:08:51.188052 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:51.188017    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6" event={"ID":"de3618a3-4b30-4e83-8ec5-5bb128cf31d4","Type":"ContainerDied","Data":"07982e643a407436bcdfcc336a93a9a6970995484906febc83b66ca6312a42f8"}
Sep 04 23:08:51.188052 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:51.188050    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6"
Sep 04 23:08:51.188281 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:08:51.188075    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07982e643a407436bcdfcc336a93a9a6970995484906febc83b66ca6312a42f8"
Sep 04 23:09:00.232069 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:00.232034    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:03.311361 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.311330    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:03.325814 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.325787    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"]
Sep 04 23:09:03.325997 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.325973    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7" podUID="5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8" containerName="payload-processing" containerID="cri-o://87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098" gracePeriod=30
Sep 04 23:09:03.411003 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.410969    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:03.512135 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.512083    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:03.512404 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.512386    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6"]
Sep 04 23:09:03.516926 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.516893    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9f64c8d3-wxpg6"]
Sep 04 23:09:03.612465 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.612424    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"
Sep 04 23:09:03.637932 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.637898    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:03.669447 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.669418    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zv4jn\" (UniqueName: \"kubernetes.io/projected/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8-kube-api-access-zv4jn\") pod \"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8\" (UID: \"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8\") "
Sep 04 23:09:03.669609 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.669489    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8-plugins-config-volume\") pod \"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8\" (UID: \"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8\") "
Sep 04 23:09:03.669926 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.669903    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8" (UID: "5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:09:03.671678 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.671631    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8-kube-api-access-zv4jn" (OuterVolumeSpecName: "kube-api-access-zv4jn") pod "5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8" (UID: "5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8"). InnerVolumeSpecName "kube-api-access-zv4jn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:09:03.770460 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.770421    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:09:03.770460 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:03.770454    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv4jn\" (UniqueName: \"kubernetes.io/projected/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8-kube-api-access-zv4jn\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:09:04.279532 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:04.279492    2578 generic.go:356] "Generic (PLEG): container finished" podID="5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8" containerID="87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098" exitCode=0
Sep 04 23:09:04.279734 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:04.279550    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"
Sep 04 23:09:04.279734 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:04.279577    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7" event={"ID":"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8","Type":"ContainerDied","Data":"87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098"}
Sep 04 23:09:04.279734 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:04.279692    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7" event={"ID":"5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8","Type":"ContainerDied","Data":"be9eca0db30b1705e8194c845119ef8792f3d4f1d246f41f5ed5c4832a8d4a26"}
Sep 04 23:09:04.279734 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:04.279722    2578 scope.go:117] "RemoveContainer" containerID="87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098"
Sep 04 23:09:04.291105 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:04.291082    2578 scope.go:117] "RemoveContainer" containerID="87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098"
Sep 04 23:09:04.291402 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:09:04.291380    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098\": container with ID starting with 87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098 not found: ID does not exist" containerID="87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098"
Sep 04 23:09:04.291474 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:04.291411    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098"} err="failed to get container status \"87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098\": rpc error: code = NotFound desc = could not find container \"87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098\": container with ID starting with 87d1b607bc8d18ed0b0456c8e6f4b45dc685a52c3cf6e4b02ef3d767626ab098 not found: ID does not exist"
Sep 04 23:09:04.304136 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:04.304112    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"]
Sep 04 23:09:04.306650 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:04.306614    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9f64c8d3-566cbf5fc7-6h5t7"]
Sep 04 23:09:04.421334 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:04.421259    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8" path="/var/lib/kubelet/pods/5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8/volumes"
Sep 04 23:09:04.421685 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:04.421566    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de3618a3-4b30-4e83-8ec5-5bb128cf31d4" path="/var/lib/kubelet/pods/de3618a3-4b30-4e83-8ec5-5bb128cf31d4/volumes"
Sep 04 23:09:06.709603 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:06.709555    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:11.792596 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:11.792561    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:12.594382 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:12.594349    2578 scope.go:117] "RemoveContainer" containerID="214cdf2da09983295415e11847220a2e46e91fd0ef77cc0d8c4cd024adfd0ada"
Sep 04 23:09:12.603192 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:12.603169    2578 scope.go:117] "RemoveContainer" containerID="3961eec30a2d8d798ca3a231759da282d40cdef5b0b37c88f03527c15a9225f0"
Sep 04 23:09:12.611300 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:12.611279    2578 scope.go:117] "RemoveContainer" containerID="c218cc61104221f213a744cb7a674263f8807700c8607152a27bc55d8bc2f9da"
Sep 04 23:09:28.182794 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:28.182760    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:28.822422 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:28.822384    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:34.322507 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:34.322465    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:38.819791 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:38.819750    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:48.548029 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:48.547988    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:48.604424 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:48.604393    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:48.791919 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:48.791885    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"]
Sep 04 23:09:48.792196 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:48.792150    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy" containerID="cri-o://25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7" gracePeriod=30
Sep 04 23:09:48.830126 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:48.830092    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:50.103299 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:50.103247    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:50.198957 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:50.198929    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:50.299710 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:50.299676    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:50.899668 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:50.899607    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:50.998870 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:50.998839    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:51.099021 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:51.098992    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:52.379534 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:52.379503    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:53.144498 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:53.144460    2578 patch_prober.go:28] interesting pod/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 04 23:09:53.144708 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:53.144528    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 04 23:09:54.056679 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.056608    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:09:54.161455 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.161427    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-envoy\") pod \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") "
Sep 04 23:09:54.161606 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.161464    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-podinfo\") pod \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") "
Sep 04 23:09:54.161606 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.161588    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-credential-socket\") pod \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") "
Sep 04 23:09:54.161825 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.161621    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-workload-socket\") pod \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") "
Sep 04 23:09:54.161825 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.161672    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istiod-ca-cert\") pod \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") "
Sep 04 23:09:54.161825 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.161697    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgv4z\" (UniqueName: \"kubernetes.io/projected/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-kube-api-access-zgv4z\") pod \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") "
Sep 04 23:09:54.161825 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.161761    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-token\") pod \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") "
Sep 04 23:09:54.161825 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.161809    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-workload-certs\") pod \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") "
Sep 04 23:09:54.162076 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.161864    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-data\") pod \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\" (UID: \"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b\") "
Sep 04 23:09:54.162076 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.161878    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" (UID: "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:09:54.162076 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.161945    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" (UID: "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:09:54.162233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.162113    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" (UID: "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:09:54.162233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.162157    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" (UID: "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:09:54.162332 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.162256    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-data" (OuterVolumeSpecName: "istio-data") pod "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" (UID: "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:09:54.162332 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.162310    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:09:54.162332 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.162325    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:09:54.162447 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.162334    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:09:54.162447 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.162343    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:09:54.163732 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.163698    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" (UID: "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:09:54.163846 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.163816    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" (UID: "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:09:54.164401 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.164372    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-kube-api-access-zgv4z" (OuterVolumeSpecName: "kube-api-access-zgv4z") pod "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" (UID: "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b"). InnerVolumeSpecName "kube-api-access-zgv4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:09:54.164401 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.164390    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-token" (OuterVolumeSpecName: "istio-token") pod "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" (UID: "10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:09:54.263102 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.263065    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:09:54.263102 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.263096    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:09:54.263102 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.263105    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:09:54.263331 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.263114    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgv4z\" (UniqueName: \"kubernetes.io/projected/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-kube-api-access-zgv4z\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:09:54.263331 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.263124    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:09:54.597711 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.597667    2578 generic.go:356] "Generic (PLEG): container finished" podID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerID="25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7" exitCode=0
Sep 04 23:09:54.597883 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.597731    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"
Sep 04 23:09:54.597883 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.597731    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" event={"ID":"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b","Type":"ContainerDied","Data":"25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7"}
Sep 04 23:09:54.597883 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.597854    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4" event={"ID":"10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b","Type":"ContainerDied","Data":"3c54e2c33c773d28502653b3c0dce8264e1f66f4a777bc058fc00b971ce6717c"}
Sep 04 23:09:54.597883 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.597879    2578 scope.go:117] "RemoveContainer" containerID="25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7"
Sep 04 23:09:54.608340 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.608319    2578 scope.go:117] "RemoveContainer" containerID="50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892"
Sep 04 23:09:54.617501 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.617479    2578 scope.go:117] "RemoveContainer" containerID="25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7"
Sep 04 23:09:54.617798 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.617777    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"]
Sep 04 23:09:54.617798 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:09:54.617786    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7\": container with ID starting with 25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7 not found: ID does not exist" containerID="25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7"
Sep 04 23:09:54.617900 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.617815    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7"} err="failed to get container status \"25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7\": rpc error: code = NotFound desc = could not find container \"25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7\": container with ID starting with 25877fc909161f31e72a4412d233cef32532eb691d816202dc6200bf5910f6d7 not found: ID does not exist"
Sep 04 23:09:54.617900 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.617832    2578 scope.go:117] "RemoveContainer" containerID="50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892"
Sep 04 23:09:54.618073 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:09:54.618055    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892\": container with ID starting with 50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892 not found: ID does not exist" containerID="50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892"
Sep 04 23:09:54.618127 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.618082    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892"} err="failed to get container status \"50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892\": rpc error: code = NotFound desc = could not find container \"50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892\": container with ID starting with 50fbe9b359b14a105393af8dd2309f37701c99604d6a5341a920a9d1b8947892 not found: ID does not exist"
Sep 04 23:09:54.621610 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:54.621588    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9f64c8d3-openshift-default-786dfdc6f6-4g2x4"]
Sep 04 23:09:56.420452 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:56.420418    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" path="/var/lib/kubelet/pods/10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b/volumes"
Sep 04 23:09:58.074864 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:58.074830    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:58.175768 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:58.175730    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:09:58.752296 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:09:58.752264    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:02.241609 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:02.241565    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:04.696759 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:04.696729    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:04.834984 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:04.834947    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:04.932456 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:04.932409    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:11.524880 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:11.524846    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:12.506123 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:12.506092    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 23:10:12.511771 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:12.511748    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 23:10:15.387596 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:15.387563    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:15.623830 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:15.623795    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:20.867102 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867065    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-8002-20b091"]
Sep 04 23:10:20.867492 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867477    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy"
Sep 04 23:10:20.867539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867494    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy"
Sep 04 23:10:20.867539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867509    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy"
Sep 04 23:10:20.867539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867516    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy"
Sep 04 23:10:20.867539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867528    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de3618a3-4b30-4e83-8ec5-5bb128cf31d4" containerName="revoke-keys"
Sep 04 23:10:20.867539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867534    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="de3618a3-4b30-4e83-8ec5-5bb128cf31d4" containerName="revoke-keys"
Sep 04 23:10:20.867771 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867541    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8" containerName="payload-processing"
Sep 04 23:10:20.867771 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867547    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8" containerName="payload-processing"
Sep 04 23:10:20.867771 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867605    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b6d5a43-4bc6-4cb0-879f-77f4ebb7fca8" containerName="payload-processing"
Sep 04 23:10:20.867771 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867614    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy"
Sep 04 23:10:20.867771 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867622    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="de3618a3-4b30-4e83-8ec5-5bb128cf31d4" containerName="revoke-keys"
Sep 04 23:10:20.867771 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.867629    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="10b1e3ff-5c8c-4b49-a3b7-c1c820e6676b" containerName="istio-proxy"
Sep 04 23:10:20.870753 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.870726    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-8002-20b091"
Sep 04 23:10:20.873368 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.873343    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-s76bh\""
Sep 04 23:10:20.877345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.877321    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-8002-20b091"]
Sep 04 23:10:20.921020 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:20.920987    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whvg4\" (UniqueName: \"kubernetes.io/projected/026fd5b7-6fcc-4631-9e1a-9bfc05ab8438-kube-api-access-whvg4\") pod \"mt-curl-8002-20b091\" (UID: \"026fd5b7-6fcc-4631-9e1a-9bfc05ab8438\") " pod="opendatahub/mt-curl-8002-20b091"
Sep 04 23:10:21.021738 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:21.021702    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whvg4\" (UniqueName: \"kubernetes.io/projected/026fd5b7-6fcc-4631-9e1a-9bfc05ab8438-kube-api-access-whvg4\") pod \"mt-curl-8002-20b091\" (UID: \"026fd5b7-6fcc-4631-9e1a-9bfc05ab8438\") " pod="opendatahub/mt-curl-8002-20b091"
Sep 04 23:10:21.030284 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:21.030253    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whvg4\" (UniqueName: \"kubernetes.io/projected/026fd5b7-6fcc-4631-9e1a-9bfc05ab8438-kube-api-access-whvg4\") pod \"mt-curl-8002-20b091\" (UID: \"026fd5b7-6fcc-4631-9e1a-9bfc05ab8438\") " pod="opendatahub/mt-curl-8002-20b091"
Sep 04 23:10:21.182699 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:21.182601    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-8002-20b091"
Sep 04 23:10:21.452912 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:21.452886    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-8002-20b091"]
Sep 04 23:10:21.454361 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:10:21.454333    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod026fd5b7_6fcc_4631_9e1a_9bfc05ab8438.slice/crio-aea27c0372d5a1e09026897de00ec08ef6364ee8f9087ffa5f2c2544842f3797 WatchSource:0}: Error finding container aea27c0372d5a1e09026897de00ec08ef6364ee8f9087ffa5f2c2544842f3797: Status 404 returned error can't find the container with id aea27c0372d5a1e09026897de00ec08ef6364ee8f9087ffa5f2c2544842f3797
Sep 04 23:10:21.455627 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:21.455610    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 23:10:21.770190 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:21.770102    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-8002-20b091" event={"ID":"026fd5b7-6fcc-4631-9e1a-9bfc05ab8438","Type":"ContainerStarted","Data":"aea27c0372d5a1e09026897de00ec08ef6364ee8f9087ffa5f2c2544842f3797"}
Sep 04 23:10:22.777715 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:22.777675    2578 generic.go:356] "Generic (PLEG): container finished" podID="026fd5b7-6fcc-4631-9e1a-9bfc05ab8438" containerID="20c1d26c6ea4fd744e92448c779f18ea08fbc956b24adb4919013715c87a5667" exitCode=0
Sep 04 23:10:22.778102 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:22.777761    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-8002-20b091" event={"ID":"026fd5b7-6fcc-4631-9e1a-9bfc05ab8438","Type":"ContainerDied","Data":"20c1d26c6ea4fd744e92448c779f18ea08fbc956b24adb4919013715c87a5667"}
Sep 04 23:10:23.925760 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:23.925737    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-8002-20b091"
Sep 04 23:10:23.944977 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:23.944957    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-8002-20b091_026fd5b7-6fcc-4631-9e1a-9bfc05ab8438/mt-curl-8002-20b091/0.log"
Sep 04 23:10:23.953178 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:23.953155    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whvg4\" (UniqueName: \"kubernetes.io/projected/026fd5b7-6fcc-4631-9e1a-9bfc05ab8438-kube-api-access-whvg4\") pod \"026fd5b7-6fcc-4631-9e1a-9bfc05ab8438\" (UID: \"026fd5b7-6fcc-4631-9e1a-9bfc05ab8438\") "
Sep 04 23:10:23.955213 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:23.955187    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/026fd5b7-6fcc-4631-9e1a-9bfc05ab8438-kube-api-access-whvg4" (OuterVolumeSpecName: "kube-api-access-whvg4") pod "026fd5b7-6fcc-4631-9e1a-9bfc05ab8438" (UID: "026fd5b7-6fcc-4631-9e1a-9bfc05ab8438"). InnerVolumeSpecName "kube-api-access-whvg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:10:23.969181 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:23.969156    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-8002-20b091"]
Sep 04 23:10:23.971580 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:23.971555    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-8002-20b091"]
Sep 04 23:10:24.054419 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.054339    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whvg4\" (UniqueName: \"kubernetes.io/projected/026fd5b7-6fcc-4631-9e1a-9bfc05ab8438-kube-api-access-whvg4\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:10:24.082748 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.082718    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-8002-3680fa"]
Sep 04 23:10:24.083136 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.083122    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026fd5b7-6fcc-4631-9e1a-9bfc05ab8438" containerName="mt-curl-8002-20b091"
Sep 04 23:10:24.083194 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.083137    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="026fd5b7-6fcc-4631-9e1a-9bfc05ab8438" containerName="mt-curl-8002-20b091"
Sep 04 23:10:24.083233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.083197    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="026fd5b7-6fcc-4631-9e1a-9bfc05ab8438" containerName="mt-curl-8002-20b091"
Sep 04 23:10:24.086190 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.086173    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-8002-3680fa"
Sep 04 23:10:24.092748 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.092726    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-8002-3680fa"]
Sep 04 23:10:24.155272 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.155239    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64zsx\" (UniqueName: \"kubernetes.io/projected/fd8e0988-419b-4a3a-ac63-95108cebf0d8-kube-api-access-64zsx\") pod \"mt-curl-8002-3680fa\" (UID: \"fd8e0988-419b-4a3a-ac63-95108cebf0d8\") " pod="opendatahub/mt-curl-8002-3680fa"
Sep 04 23:10:24.256665 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.256592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64zsx\" (UniqueName: \"kubernetes.io/projected/fd8e0988-419b-4a3a-ac63-95108cebf0d8-kube-api-access-64zsx\") pod \"mt-curl-8002-3680fa\" (UID: \"fd8e0988-419b-4a3a-ac63-95108cebf0d8\") " pod="opendatahub/mt-curl-8002-3680fa"
Sep 04 23:10:24.265756 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.265723    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64zsx\" (UniqueName: \"kubernetes.io/projected/fd8e0988-419b-4a3a-ac63-95108cebf0d8-kube-api-access-64zsx\") pod \"mt-curl-8002-3680fa\" (UID: \"fd8e0988-419b-4a3a-ac63-95108cebf0d8\") " pod="opendatahub/mt-curl-8002-3680fa"
Sep 04 23:10:24.396548 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.396512    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-8002-3680fa"
Sep 04 23:10:24.420849 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.420818    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="026fd5b7-6fcc-4631-9e1a-9bfc05ab8438" path="/var/lib/kubelet/pods/026fd5b7-6fcc-4631-9e1a-9bfc05ab8438/volumes"
Sep 04 23:10:24.670764 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.670739    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-8002-3680fa"]
Sep 04 23:10:24.671979 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:10:24.671950    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd8e0988_419b_4a3a_ac63_95108cebf0d8.slice/crio-e9f493ad573d36f5555916f0b991988ef40fb9a74d5ac941740444de99cb4192 WatchSource:0}: Error finding container e9f493ad573d36f5555916f0b991988ef40fb9a74d5ac941740444de99cb4192: Status 404 returned error can't find the container with id e9f493ad573d36f5555916f0b991988ef40fb9a74d5ac941740444de99cb4192
Sep 04 23:10:24.793919 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.793877    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-8002-3680fa" event={"ID":"fd8e0988-419b-4a3a-ac63-95108cebf0d8","Type":"ContainerStarted","Data":"e9f493ad573d36f5555916f0b991988ef40fb9a74d5ac941740444de99cb4192"}
Sep 04 23:10:24.795587 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.795557    2578 scope.go:117] "RemoveContainer" containerID="20c1d26c6ea4fd744e92448c779f18ea08fbc956b24adb4919013715c87a5667"
Sep 04 23:10:24.795587 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:24.795572    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-8002-20b091"
Sep 04 23:10:25.802500 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:25.802462    2578 generic.go:356] "Generic (PLEG): container finished" podID="fd8e0988-419b-4a3a-ac63-95108cebf0d8" containerID="2d68b50ff4412f4d05357088eb9d0ed72c98d8e6cd8b2ad05d79a13883579cea" exitCode=0
Sep 04 23:10:25.802931 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:25.802563    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-8002-3680fa" event={"ID":"fd8e0988-419b-4a3a-ac63-95108cebf0d8","Type":"ContainerDied","Data":"2d68b50ff4412f4d05357088eb9d0ed72c98d8e6cd8b2ad05d79a13883579cea"}
Sep 04 23:10:26.940958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:26.940934    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-8002-3680fa"
Sep 04 23:10:26.958155 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:26.958129    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-8002-3680fa_fd8e0988-419b-4a3a-ac63-95108cebf0d8/mt-curl-8002-3680fa/0.log"
Sep 04 23:10:26.978968 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:26.978941    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-8002-3680fa"]
Sep 04 23:10:26.981462 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:26.981441    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-8002-3680fa"]
Sep 04 23:10:26.983961 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:26.983943    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64zsx\" (UniqueName: \"kubernetes.io/projected/fd8e0988-419b-4a3a-ac63-95108cebf0d8-kube-api-access-64zsx\") pod \"fd8e0988-419b-4a3a-ac63-95108cebf0d8\" (UID: \"fd8e0988-419b-4a3a-ac63-95108cebf0d8\") "
Sep 04 23:10:26.985954 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:26.985933    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd8e0988-419b-4a3a-ac63-95108cebf0d8-kube-api-access-64zsx" (OuterVolumeSpecName: "kube-api-access-64zsx") pod "fd8e0988-419b-4a3a-ac63-95108cebf0d8" (UID: "fd8e0988-419b-4a3a-ac63-95108cebf0d8"). InnerVolumeSpecName "kube-api-access-64zsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:10:27.084621 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:27.084586    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64zsx\" (UniqueName: \"kubernetes.io/projected/fd8e0988-419b-4a3a-ac63-95108cebf0d8-kube-api-access-64zsx\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:10:27.816878 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:27.816847    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-8002-3680fa"
Sep 04 23:10:27.816878 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:27.816871    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9f493ad573d36f5555916f0b991988ef40fb9a74d5ac941740444de99cb4192"
Sep 04 23:10:27.826766 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:10:27.826737    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-8002-3680fa\" is forbidden: User \"system:node:ip-10-0-128-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-32.ec2.internal' and this object" podUID="fd8e0988-419b-4a3a-ac63-95108cebf0d8" pod="opendatahub/mt-curl-8002-3680fa"
Sep 04 23:10:28.420002 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:28.419966    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd8e0988-419b-4a3a-ac63-95108cebf0d8" path="/var/lib/kubelet/pods/fd8e0988-419b-4a3a-ac63-95108cebf0d8/volumes"
Sep 04 23:10:28.986431 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:28.986386    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:30.028683 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:30.028635    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:31.538938 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:31.538903    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:36.547052 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:36.546978    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:42.371300 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:42.371264    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:42.430364 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:42.430335    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:42.753273 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:42.753176    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:43.042979 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.042884    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"]
Sep 04 23:10:43.043574 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.043552    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd8e0988-419b-4a3a-ac63-95108cebf0d8" containerName="mt-curl-8002-3680fa"
Sep 04 23:10:43.043678 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.043576    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd8e0988-419b-4a3a-ac63-95108cebf0d8" containerName="mt-curl-8002-3680fa"
Sep 04 23:10:43.043733 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.043706    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd8e0988-419b-4a3a-ac63-95108cebf0d8" containerName="mt-curl-8002-3680fa"
Sep 04 23:10:43.047603 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.047583    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.050058 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.050033    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-17eaa7-openshift-default-dockercfg-jlk6s\""
Sep 04 23:10:43.058855 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.058830    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"]
Sep 04 23:10:43.080229 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.080179    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:43.141217 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.141182    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-podinfo\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.141410 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.141288    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f577ede7-e488-4160-ab2d-ee551c7f1a97-istiod-ca-cert\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.141410 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.141327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j9td\" (UniqueName: \"kubernetes.io/projected/f577ede7-e488-4160-ab2d-ee551c7f1a97-kube-api-access-5j9td\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.141410 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.141358    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-workload-certs\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.141410 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.141391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-token\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.141601 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.141455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-envoy\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.141601 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.141522    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-data\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.141601 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.141560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-credential-socket\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.141601 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.141582    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-workload-socket\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.176140 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.176095    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:43.242804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.242763    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f577ede7-e488-4160-ab2d-ee551c7f1a97-istiod-ca-cert\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.242804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.242811    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j9td\" (UniqueName: \"kubernetes.io/projected/f577ede7-e488-4160-ab2d-ee551c7f1a97-kube-api-access-5j9td\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.243071 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.242840    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-workload-certs\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.243071 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.242869    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-token\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.243071 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.242907    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-envoy\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.243071 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.243022    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-data\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.243290 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.243083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-credential-socket\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.243290 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.243134    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-workload-socket\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.243290 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.243231    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-podinfo\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.243530 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.243502    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-credential-socket\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.243661 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.243593    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-workload-certs\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.243661 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.243599    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f577ede7-e488-4160-ab2d-ee551c7f1a97-istiod-ca-cert\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.243939 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.243912    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-workload-socket\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.244040 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.243949    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-data\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.245456 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.245429    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-envoy\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.245672 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.245635    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-podinfo\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.250841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.250817    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-token\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.251023 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.251008    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j9td\" (UniqueName: \"kubernetes.io/projected/f577ede7-e488-4160-ab2d-ee551c7f1a97-kube-api-access-5j9td\") pod \"e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") " pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.360523 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.360487    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:43.639400 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.639365    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"]
Sep 04 23:10:43.640356 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:10:43.640325    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf577ede7_e488_4160_ab2d_ee551c7f1a97.slice/crio-09552dc41fb8878d04eacebe7c7c3e583f39e389baad9431e29584bbccabc4b7 WatchSource:0}: Error finding container 09552dc41fb8878d04eacebe7c7c3e583f39e389baad9431e29584bbccabc4b7: Status 404 returned error can't find the container with id 09552dc41fb8878d04eacebe7c7c3e583f39e389baad9431e29584bbccabc4b7
Sep 04 23:10:43.643302 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.643259    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:10:43.643423 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.643358    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:10:43.643423 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.643401    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:10:43.921537 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.921447    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" event={"ID":"f577ede7-e488-4160-ab2d-ee551c7f1a97","Type":"ContainerStarted","Data":"632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0"}
Sep 04 23:10:43.921537 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.921513    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" event={"ID":"f577ede7-e488-4160-ab2d-ee551c7f1a97","Type":"ContainerStarted","Data":"09552dc41fb8878d04eacebe7c7c3e583f39e389baad9431e29584bbccabc4b7"}
Sep 04 23:10:43.944415 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:43.944367    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" podStartSLOduration=0.944352249 podStartE2EDuration="944.352249ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:10:43.940817371 +0000 UTC m=+1532.050937957" watchObservedRunningTime="2026-09-04 23:10:43.944352249 +0000 UTC m=+1532.054472834"
Sep 04 23:10:44.361098 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:44.361055    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:44.366188 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:44.366162    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:44.931936 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:44.931901    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:44.933589 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:44.933559    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:10:45.901333 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:45.901298    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:45.999789 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:45.999753    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:47.041150 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:47.041117    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:47.141316 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:47.141275    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:47.254088 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:47.254044    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:47.544067 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:47.544034    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:47.605214 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:47.605183    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:53.959139 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:53.959101    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:10:54.062173 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:10:54.062132    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:00.972863 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:00.972812    2578 patch_prober.go:28] interesting pod/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 04 23:11:00.973405 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:00.972954    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 04 23:11:02.050583 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:02.050548    2578 generic.go:356] "Generic (PLEG): container finished" podID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerID="632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0" exitCode=0
Sep 04 23:11:02.051036 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:02.050628    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" event={"ID":"f577ede7-e488-4160-ab2d-ee551c7f1a97","Type":"ContainerDied","Data":"632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0"}
Sep 04 23:11:02.051141 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:02.051126    2578 scope.go:117] "RemoveContainer" containerID="632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0"
Sep 04 23:11:02.052289 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:02.052255    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:11:02.052394 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:02.052337    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:11:02.052394 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:02.052386    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:11:03.063826 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:03.063784    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" event={"ID":"f577ede7-e488-4160-ab2d-ee551c7f1a97","Type":"ContainerStarted","Data":"7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f"}
Sep 04 23:11:03.360925 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:03.360875    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:11:03.361161 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:03.361138    2578 patch_prober.go:28] interesting pod/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 04 23:11:03.361236 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:03.361195    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 04 23:11:03.927948 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:03.927915    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:04.361165 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:04.361120    2578 patch_prober.go:28] interesting pod/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 04 23:11:04.361709 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:04.361198    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 04 23:11:05.361678 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:05.361613    2578 patch_prober.go:28] interesting pod/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 04 23:11:05.362152 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:05.361706    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 04 23:11:06.361315 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:06.361272    2578 patch_prober.go:28] interesting pod/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 04 23:11:06.361517 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:06.361352    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 04 23:11:07.361526 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:07.361478    2578 patch_prober.go:28] interesting pod/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 04 23:11:07.361931 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:07.361572    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 04 23:11:08.501728 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:08.501695    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:11:08.502205 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:08.502089    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:11:08.502804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:08.502783    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:11:10.886912 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:10.886869    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:11.365760 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.365726    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:11.660062 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.659973    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"]
Sep 04 23:11:11.664629 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.664605    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.667214 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.667189    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9b6db4-openshift-default-dockercfg-459tb\""
Sep 04 23:11:11.674740 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.674715    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"]
Sep 04 23:11:11.698891 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.698858    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:11.738429 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.738393    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-workload-socket\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.738611 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.738454    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istiod-ca-cert\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.738611 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.738495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-envoy\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.738611 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.738546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2289m\" (UniqueName: \"kubernetes.io/projected/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-kube-api-access-2289m\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.738771 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.738627    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-credential-socket\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.738771 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.738694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-data\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.738771 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.738721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-token\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.738876 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.738810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-podinfo\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.738919 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.738894    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-workload-certs\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840017 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.839980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-data\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840017 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.840022    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-token\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.840167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-podinfo\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.840300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-workload-certs\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840409 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.840347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-workload-socket\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840409 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.840392    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istiod-ca-cert\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840506 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.840410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-data\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840506 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.840424    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-envoy\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840600 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.840502    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2289m\" (UniqueName: \"kubernetes.io/projected/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-kube-api-access-2289m\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840600 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.840557    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-credential-socket\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840724 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.840629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-workload-certs\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840724 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.840711    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-workload-socket\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.840887 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.840860    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-credential-socket\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.841229 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.841208    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istiod-ca-cert\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.843257 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.843237    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-envoy\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.843468 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.843448    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-podinfo\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.848141 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.848117    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-token\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.848444 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.848422    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2289m\" (UniqueName: \"kubernetes.io/projected/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-kube-api-access-2289m\") pod \"e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") " pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:11.976247 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:11.976134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:12.095902 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:12.095871    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:12.252966 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:12.252939    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"]
Sep 04 23:11:12.254024 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:11:12.253999    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff08f0ae_d44d_424b_92a4_1ce69d2b17f4.slice/crio-18fb0f0aeef6b2a5f2704612c7ab0d00a20b0159cce520770d5df5991646312c WatchSource:0}: Error finding container 18fb0f0aeef6b2a5f2704612c7ab0d00a20b0159cce520770d5df5991646312c: Status 404 returned error can't find the container with id 18fb0f0aeef6b2a5f2704612c7ab0d00a20b0159cce520770d5df5991646312c
Sep 04 23:11:12.256290 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:12.256257    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:11:12.256359 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:12.256328    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:11:12.256399 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:12.256356    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:11:12.659707 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:12.659670    2578 scope.go:117] "RemoveContainer" containerID="71e4b577f372de15e948602465b32221224903efc47378594ac55667ba3d3a5a"
Sep 04 23:11:13.127064 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:13.127028    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" event={"ID":"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4","Type":"ContainerStarted","Data":"fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e"}
Sep 04 23:11:13.127558 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:13.127087    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" event={"ID":"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4","Type":"ContainerStarted","Data":"18fb0f0aeef6b2a5f2704612c7ab0d00a20b0159cce520770d5df5991646312c"}
Sep 04 23:11:13.147880 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:13.147815    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" podStartSLOduration=2.147794493 podStartE2EDuration="2.147794493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:11:13.145191208 +0000 UTC m=+1561.255311796" watchObservedRunningTime="2026-09-04 23:11:13.147794493 +0000 UTC m=+1561.257915083"
Sep 04 23:11:13.976503 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:13.976453    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:13.981957 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:13.981926    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:14.133113 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:14.133076    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:14.134087 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:14.134063    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:15.084036 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:15.084006    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:15.178350 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:15.178318    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:16.354891 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:16.354859    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:16.488699 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:16.488633    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:16.613416 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:16.613332    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:16.779196 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:16.779165    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:16.881618 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:16.881517    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:16.920868 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:16.920835    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"]
Sep 04 23:11:16.925189 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:16.925167    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"
Sep 04 23:11:16.927852 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:16.927827    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-9b6db4\""
Sep 04 23:11:16.927990 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:16.927920    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-9b6db4-dockercfg-q9kg7\""
Sep 04 23:11:16.934978 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:16.934423    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"]
Sep 04 23:11:17.003567 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:17.002198    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d009f8d9-87f5-4154-9ee1-3addc84e73ec-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj\" (UID: \"d009f8d9-87f5-4154-9ee1-3addc84e73ec\") " pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"
Sep 04 23:11:17.003567 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:17.002377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdg6k\" (UniqueName: \"kubernetes.io/projected/d009f8d9-87f5-4154-9ee1-3addc84e73ec-kube-api-access-gdg6k\") pod \"payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj\" (UID: \"d009f8d9-87f5-4154-9ee1-3addc84e73ec\") " pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"
Sep 04 23:11:17.103634 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:17.103603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdg6k\" (UniqueName: \"kubernetes.io/projected/d009f8d9-87f5-4154-9ee1-3addc84e73ec-kube-api-access-gdg6k\") pod \"payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj\" (UID: \"d009f8d9-87f5-4154-9ee1-3addc84e73ec\") " pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"
Sep 04 23:11:17.103812 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:17.103689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d009f8d9-87f5-4154-9ee1-3addc84e73ec-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj\" (UID: \"d009f8d9-87f5-4154-9ee1-3addc84e73ec\") " pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"
Sep 04 23:11:17.104393 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:17.104371    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d009f8d9-87f5-4154-9ee1-3addc84e73ec-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj\" (UID: \"d009f8d9-87f5-4154-9ee1-3addc84e73ec\") " pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"
Sep 04 23:11:17.111368 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:17.111344    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdg6k\" (UniqueName: \"kubernetes.io/projected/d009f8d9-87f5-4154-9ee1-3addc84e73ec-kube-api-access-gdg6k\") pod \"payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj\" (UID: \"d009f8d9-87f5-4154-9ee1-3addc84e73ec\") " pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"
Sep 04 23:11:17.238993 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:17.238904    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"
Sep 04 23:11:17.512403 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:11:17.512374    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd009f8d9_87f5_4154_9ee1_3addc84e73ec.slice/crio-5bf58af017dad06a538558ee1b234c0714b546d63b39716f7a853262be1e88e1 WatchSource:0}: Error finding container 5bf58af017dad06a538558ee1b234c0714b546d63b39716f7a853262be1e88e1: Status 404 returned error can't find the container with id 5bf58af017dad06a538558ee1b234c0714b546d63b39716f7a853262be1e88e1
Sep 04 23:11:17.512746 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:17.512612    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"]
Sep 04 23:11:18.161003 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:18.160963    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj" event={"ID":"d009f8d9-87f5-4154-9ee1-3addc84e73ec","Type":"ContainerStarted","Data":"92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246"}
Sep 04 23:11:18.161202 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:18.161030    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj" event={"ID":"d009f8d9-87f5-4154-9ee1-3addc84e73ec","Type":"ContainerStarted","Data":"5bf58af017dad06a538558ee1b234c0714b546d63b39716f7a853262be1e88e1"}
Sep 04 23:11:18.161202 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:18.161060    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"
Sep 04 23:11:18.178442 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:18.178390    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj" podStartSLOduration=2.1783740639999998 podStartE2EDuration="2.178374064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:11:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:11:18.17624344 +0000 UTC m=+1566.286364025" watchObservedRunningTime="2026-09-04 23:11:18.178374064 +0000 UTC m=+1566.288494675"
Sep 04 23:11:18.420264 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:18.420168    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:29.170342 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:29.170309    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"
Sep 04 23:11:34.269682 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:34.269628    2578 generic.go:356] "Generic (PLEG): container finished" podID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerID="fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e" exitCode=0
Sep 04 23:11:34.270080 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:34.269705    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" event={"ID":"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4","Type":"ContainerDied","Data":"fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e"}
Sep 04 23:11:34.270181 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:34.270166    2578 scope.go:117] "RemoveContainer" containerID="fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e"
Sep 04 23:11:34.271356 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:34.271325    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:11:34.271409 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:34.271394    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:11:34.271451 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:34.271426    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:11:35.279486 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:35.279455    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" event={"ID":"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4","Type":"ContainerStarted","Data":"fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f"}
Sep 04 23:11:35.977124 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:35.977083    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:35.977305 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:35.977279    2578 patch_prober.go:28] interesting pod/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 04 23:11:35.977355 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:35.977319    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 04 23:11:36.976627 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:36.976585    2578 patch_prober.go:28] interesting pod/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 04 23:11:36.977041 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:36.976666    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 04 23:11:37.179683 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:37.179635    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:37.977054 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:37.977012    2578 patch_prober.go:28] interesting pod/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 04 23:11:37.977446 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:37.977089    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 04 23:11:38.977083 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:38.977041    2578 patch_prober.go:28] interesting pod/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 04 23:11:38.977451 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:38.977110    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 04 23:11:39.976435 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:39.976395    2578 patch_prober.go:28] interesting pod/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 04 23:11:39.976596 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:39.976449    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 04 23:11:40.065744 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:40.065700    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:40.981147 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:40.981113    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:40.981404 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:40.981384    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:40.982140 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:40.982120    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:11:43.184223 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:43.184185    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb"]
Sep 04 23:11:43.189112 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:43.189095    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb"
Sep 04 23:11:43.192210 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:43.192170    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:11:43.196113 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:43.196089    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb"]
Sep 04 23:11:43.280654 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:43.280607    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3519936-dc95-4dd3-8b08-b0d629fa829e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb\" (UID: \"c3519936-dc95-4dd3-8b08-b0d629fa829e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb"
Sep 04 23:11:43.382293 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:43.382253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3519936-dc95-4dd3-8b08-b0d629fa829e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb\" (UID: \"c3519936-dc95-4dd3-8b08-b0d629fa829e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb"
Sep 04 23:11:43.383558 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:43.383428    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3519936-dc95-4dd3-8b08-b0d629fa829e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb\" (UID: \"c3519936-dc95-4dd3-8b08-b0d629fa829e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb"
Sep 04 23:11:43.387154 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:43.387125    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:43.501286 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:43.501201    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb"
Sep 04 23:11:43.773201 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:43.773174    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb"]
Sep 04 23:11:43.774949 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:11:43.774927    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3519936_dc95_4dd3_8b08_b0d629fa829e.slice/crio-b9d1bd567f887055799c3ab636314e7af55bfb225e067ce5548d345d4ce91262 WatchSource:0}: Error finding container b9d1bd567f887055799c3ab636314e7af55bfb225e067ce5548d345d4ce91262: Status 404 returned error can't find the container with id b9d1bd567f887055799c3ab636314e7af55bfb225e067ce5548d345d4ce91262
Sep 04 23:11:44.340226 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:44.340171    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb" event={"ID":"c3519936-dc95-4dd3-8b08-b0d629fa829e","Type":"ContainerStarted","Data":"cd6be43a6cce8e7b1e4a9be28782d6d9eca48c36b59fedc6111793ca439ba2a4"}
Sep 04 23:11:44.340612 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:44.340259    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb" event={"ID":"c3519936-dc95-4dd3-8b08-b0d629fa829e","Type":"ContainerStarted","Data":"b9d1bd567f887055799c3ab636314e7af55bfb225e067ce5548d345d4ce91262"}
Sep 04 23:11:44.355271 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:44.355217    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb" podStartSLOduration=1.355204246 podStartE2EDuration="1.355204246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:11:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:11:44.353660803 +0000 UTC m=+1592.463781389" watchObservedRunningTime="2026-09-04 23:11:44.355204246 +0000 UTC m=+1592.465324832"
Sep 04 23:11:45.348467 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:45.348427    2578 generic.go:356] "Generic (PLEG): container finished" podID="c3519936-dc95-4dd3-8b08-b0d629fa829e" containerID="cd6be43a6cce8e7b1e4a9be28782d6d9eca48c36b59fedc6111793ca439ba2a4" exitCode=0
Sep 04 23:11:45.348894 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:45.348503    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb" event={"ID":"c3519936-dc95-4dd3-8b08-b0d629fa829e","Type":"ContainerDied","Data":"cd6be43a6cce8e7b1e4a9be28782d6d9eca48c36b59fedc6111793ca439ba2a4"}
Sep 04 23:11:46.013156 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:46.013116    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:46.511443 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:46.511418    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb"
Sep 04 23:11:46.616228 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:46.616133    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3519936-dc95-4dd3-8b08-b0d629fa829e-maas-api-service-ca\") pod \"c3519936-dc95-4dd3-8b08-b0d629fa829e\" (UID: \"c3519936-dc95-4dd3-8b08-b0d629fa829e\") "
Sep 04 23:11:46.616492 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:46.616467    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3519936-dc95-4dd3-8b08-b0d629fa829e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c3519936-dc95-4dd3-8b08-b0d629fa829e" (UID: "c3519936-dc95-4dd3-8b08-b0d629fa829e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:11:46.720946 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:46.720913    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3519936-dc95-4dd3-8b08-b0d629fa829e-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:11:47.362912 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:47.362880    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb"
Sep 04 23:11:47.363080 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:47.362894    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb" event={"ID":"c3519936-dc95-4dd3-8b08-b0d629fa829e","Type":"ContainerDied","Data":"b9d1bd567f887055799c3ab636314e7af55bfb225e067ce5548d345d4ce91262"}
Sep 04 23:11:47.363080 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:47.363041    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9d1bd567f887055799c3ab636314e7af55bfb225e067ce5548d345d4ce91262"
Sep 04 23:11:47.850736 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:47.850687    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:53.141742 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:53.141705    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:53.320670 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:53.320610    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:58.931267 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:58.931219    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:59.004980 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:59.004934    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:59.063387 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:59.063358    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:59.141084 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:59.141048    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb"]
Sep 04 23:11:59.142300 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:59.142264    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:59.143755 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:59.143734    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-17eaa7-ps2qb"]
Sep 04 23:11:59.200452 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:59.200380    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:11:59.532118 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:11:59.532023    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:00.419902 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:00.419869    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3519936-dc95-4dd3-8b08-b0d629fa829e" path="/var/lib/kubelet/pods/c3519936-dc95-4dd3-8b08-b0d629fa829e/volumes"
Sep 04 23:12:05.284664 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:05.284571    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:05.385772 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:05.385739    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:05.516800 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:05.516772    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"]
Sep 04 23:12:05.516991 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:05.516967    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy" containerID="cri-o://7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f" gracePeriod=30
Sep 04 23:12:05.537059 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:05.536975    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:06.012054 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:06.012016    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:08.504820 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:08.504780    2578 patch_prober.go:28] interesting pod/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 04 23:12:08.505262 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:08.504840    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 04 23:12:10.778415 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.778387    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:12:10.967945 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.967912    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-podinfo\") pod \"f577ede7-e488-4160-ab2d-ee551c7f1a97\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") "
Sep 04 23:12:10.968126 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.967987    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-workload-certs\") pod \"f577ede7-e488-4160-ab2d-ee551c7f1a97\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") "
Sep 04 23:12:10.968126 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968048    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j9td\" (UniqueName: \"kubernetes.io/projected/f577ede7-e488-4160-ab2d-ee551c7f1a97-kube-api-access-5j9td\") pod \"f577ede7-e488-4160-ab2d-ee551c7f1a97\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") "
Sep 04 23:12:10.968126 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968102    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-envoy\") pod \"f577ede7-e488-4160-ab2d-ee551c7f1a97\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") "
Sep 04 23:12:10.968291 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968124    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f577ede7-e488-4160-ab2d-ee551c7f1a97-istiod-ca-cert\") pod \"f577ede7-e488-4160-ab2d-ee551c7f1a97\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") "
Sep 04 23:12:10.968291 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968150    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-token\") pod \"f577ede7-e488-4160-ab2d-ee551c7f1a97\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") "
Sep 04 23:12:10.968291 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968185    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-workload-socket\") pod \"f577ede7-e488-4160-ab2d-ee551c7f1a97\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") "
Sep 04 23:12:10.968291 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968245    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-credential-socket\") pod \"f577ede7-e488-4160-ab2d-ee551c7f1a97\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") "
Sep 04 23:12:10.968291 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968282    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-data\") pod \"f577ede7-e488-4160-ab2d-ee551c7f1a97\" (UID: \"f577ede7-e488-4160-ab2d-ee551c7f1a97\") "
Sep 04 23:12:10.968553 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968330    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f577ede7-e488-4160-ab2d-ee551c7f1a97" (UID: "f577ede7-e488-4160-ab2d-ee551c7f1a97"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:10.968616 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968565    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f577ede7-e488-4160-ab2d-ee551c7f1a97" (UID: "f577ede7-e488-4160-ab2d-ee551c7f1a97"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:10.968616 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968597    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f577ede7-e488-4160-ab2d-ee551c7f1a97" (UID: "f577ede7-e488-4160-ab2d-ee551c7f1a97"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:10.968769 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968632    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f577ede7-e488-4160-ab2d-ee551c7f1a97-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f577ede7-e488-4160-ab2d-ee551c7f1a97" (UID: "f577ede7-e488-4160-ab2d-ee551c7f1a97"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:12:10.968830 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968795    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-data" (OuterVolumeSpecName: "istio-data") pod "f577ede7-e488-4160-ab2d-ee551c7f1a97" (UID: "f577ede7-e488-4160-ab2d-ee551c7f1a97"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:10.968946 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968920    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:10.968946 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968945    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:10.969149 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.968956    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:10.970172 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.970149    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f577ede7-e488-4160-ab2d-ee551c7f1a97" (UID: "f577ede7-e488-4160-ab2d-ee551c7f1a97"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:12:10.970349 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.970328    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f577ede7-e488-4160-ab2d-ee551c7f1a97" (UID: "f577ede7-e488-4160-ab2d-ee551c7f1a97"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:10.970463 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.970441    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f577ede7-e488-4160-ab2d-ee551c7f1a97-kube-api-access-5j9td" (OuterVolumeSpecName: "kube-api-access-5j9td") pod "f577ede7-e488-4160-ab2d-ee551c7f1a97" (UID: "f577ede7-e488-4160-ab2d-ee551c7f1a97"). InnerVolumeSpecName "kube-api-access-5j9td". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:12:10.970541 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:10.970512    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-token" (OuterVolumeSpecName: "istio-token") pod "f577ede7-e488-4160-ab2d-ee551c7f1a97" (UID: "f577ede7-e488-4160-ab2d-ee551c7f1a97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:12:11.069624 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.069585    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j9td\" (UniqueName: \"kubernetes.io/projected/f577ede7-e488-4160-ab2d-ee551c7f1a97-kube-api-access-5j9td\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:11.069624 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.069618    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:11.069624 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.069627    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f577ede7-e488-4160-ab2d-ee551c7f1a97-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:11.069897 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.069635    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:11.069897 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.069675    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:11.069897 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.069683    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f577ede7-e488-4160-ab2d-ee551c7f1a97-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:11.515362 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.515328    2578 generic.go:356] "Generic (PLEG): container finished" podID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerID="7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f" exitCode=0
Sep 04 23:12:11.515531 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.515381    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"
Sep 04 23:12:11.515531 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.515409    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" event={"ID":"f577ede7-e488-4160-ab2d-ee551c7f1a97","Type":"ContainerDied","Data":"7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f"}
Sep 04 23:12:11.515531 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.515463    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2" event={"ID":"f577ede7-e488-4160-ab2d-ee551c7f1a97","Type":"ContainerDied","Data":"09552dc41fb8878d04eacebe7c7c3e583f39e389baad9431e29584bbccabc4b7"}
Sep 04 23:12:11.515531 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.515485    2578 scope.go:117] "RemoveContainer" containerID="7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f"
Sep 04 23:12:11.526472 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.526454    2578 scope.go:117] "RemoveContainer" containerID="632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0"
Sep 04 23:12:11.535298 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.535276    2578 scope.go:117] "RemoveContainer" containerID="7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f"
Sep 04 23:12:11.535692 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:12:11.535572    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f\": container with ID starting with 7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f not found: ID does not exist" containerID="7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f"
Sep 04 23:12:11.535781 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.535706    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f"} err="failed to get container status \"7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f\": rpc error: code = NotFound desc = could not find container \"7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f\": container with ID starting with 7411a626854e0f05866916294bf115370837c090b79fdae8fec6f68b1c1dcc2f not found: ID does not exist"
Sep 04 23:12:11.535781 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.535735    2578 scope.go:117] "RemoveContainer" containerID="632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0"
Sep 04 23:12:11.536457 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:12:11.536380    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0\": container with ID starting with 632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0 not found: ID does not exist" containerID="632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0"
Sep 04 23:12:11.536457 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.536421    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0"} err="failed to get container status \"632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0\": rpc error: code = NotFound desc = could not find container \"632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0\": container with ID starting with 632cf13f1779818f9240c775b53ec003d15a69bc81899eb09f0c7ad57ae965e0 not found: ID does not exist"
Sep 04 23:12:11.541510 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.541489    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"]
Sep 04 23:12:11.544193 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:11.544166    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-17eaa7-openshift-default-797d8bcfb8-6thr2"]
Sep 04 23:12:12.421183 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:12.421146    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" path="/var/lib/kubelet/pods/f577ede7-e488-4160-ab2d-ee551c7f1a97/volumes"
Sep 04 23:12:12.690696 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:12.690613    2578 scope.go:117] "RemoveContainer" containerID="54b6bbb7e4d45f27bf06ecb3572d79971fa083080feb38b9db5d7fe3c20a4fd7"
Sep 04 23:12:21.478057 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.477993    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:21.504126 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.504087    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"]
Sep 04 23:12:21.504410 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.504355    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj" podUID="d009f8d9-87f5-4154-9ee1-3addc84e73ec" containerName="payload-processing" containerID="cri-o://92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246" gracePeriod=30
Sep 04 23:12:21.578562 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.578536    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:21.668434 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.668387    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:21.723333 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.723303    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:21.787527 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.787496    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"
Sep 04 23:12:21.875011 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.874973    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdg6k\" (UniqueName: \"kubernetes.io/projected/d009f8d9-87f5-4154-9ee1-3addc84e73ec-kube-api-access-gdg6k\") pod \"d009f8d9-87f5-4154-9ee1-3addc84e73ec\" (UID: \"d009f8d9-87f5-4154-9ee1-3addc84e73ec\") "
Sep 04 23:12:21.875194 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.875072    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d009f8d9-87f5-4154-9ee1-3addc84e73ec-plugins-config-volume\") pod \"d009f8d9-87f5-4154-9ee1-3addc84e73ec\" (UID: \"d009f8d9-87f5-4154-9ee1-3addc84e73ec\") "
Sep 04 23:12:21.875478 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.875449    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d009f8d9-87f5-4154-9ee1-3addc84e73ec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d009f8d9-87f5-4154-9ee1-3addc84e73ec" (UID: "d009f8d9-87f5-4154-9ee1-3addc84e73ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:12:21.877021 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.876997    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d009f8d9-87f5-4154-9ee1-3addc84e73ec-kube-api-access-gdg6k" (OuterVolumeSpecName: "kube-api-access-gdg6k") pod "d009f8d9-87f5-4154-9ee1-3addc84e73ec" (UID: "d009f8d9-87f5-4154-9ee1-3addc84e73ec"). InnerVolumeSpecName "kube-api-access-gdg6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:12:21.976151 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.976117    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdg6k\" (UniqueName: \"kubernetes.io/projected/d009f8d9-87f5-4154-9ee1-3addc84e73ec-kube-api-access-gdg6k\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:21.976151 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:21.976146    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d009f8d9-87f5-4154-9ee1-3addc84e73ec-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:22.029201 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:22.029091    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:22.592425 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:22.592385    2578 generic.go:356] "Generic (PLEG): container finished" podID="d009f8d9-87f5-4154-9ee1-3addc84e73ec" containerID="92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246" exitCode=0
Sep 04 23:12:22.593192 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:22.592435    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"
Sep 04 23:12:22.593192 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:22.592478    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj" event={"ID":"d009f8d9-87f5-4154-9ee1-3addc84e73ec","Type":"ContainerDied","Data":"92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246"}
Sep 04 23:12:22.593192 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:22.592541    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj" event={"ID":"d009f8d9-87f5-4154-9ee1-3addc84e73ec","Type":"ContainerDied","Data":"5bf58af017dad06a538558ee1b234c0714b546d63b39716f7a853262be1e88e1"}
Sep 04 23:12:22.593192 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:22.592578    2578 scope.go:117] "RemoveContainer" containerID="92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246"
Sep 04 23:12:22.602660 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:22.602625    2578 scope.go:117] "RemoveContainer" containerID="92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246"
Sep 04 23:12:22.602912 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:12:22.602891    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246\": container with ID starting with 92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246 not found: ID does not exist" containerID="92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246"
Sep 04 23:12:22.602966 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:22.602920    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246"} err="failed to get container status \"92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246\": rpc error: code = NotFound desc = could not find container \"92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246\": container with ID starting with 92bf9573efa34581f06bc79915c4b58301ddf87010fc3f977e15246dbeee6246 not found: ID does not exist"
Sep 04 23:12:22.610669 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:22.610624    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"]
Sep 04 23:12:22.614755 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:22.614734    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9b6db4-5668757455-f8bcj"]
Sep 04 23:12:24.420159 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:24.420128    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d009f8d9-87f5-4154-9ee1-3addc84e73ec" path="/var/lib/kubelet/pods/d009f8d9-87f5-4154-9ee1-3addc84e73ec/volumes"
Sep 04 23:12:27.424950 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:27.424914    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:27.526345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:27.526312    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:28.026369 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.026339    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:28.127237 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.127208    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:28.249605 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.249569    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"]
Sep 04 23:12:28.249865 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.249837    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerName="istio-proxy" containerID="cri-o://fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f" gracePeriod=30
Sep 04 23:12:28.264916 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.264888    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:28.641164 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.641128    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f"]
Sep 04 23:12:28.641792 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.641771    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy"
Sep 04 23:12:28.641860 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.641796    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy"
Sep 04 23:12:28.641860 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.641826    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d009f8d9-87f5-4154-9ee1-3addc84e73ec" containerName="payload-processing"
Sep 04 23:12:28.641860 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.641835    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d009f8d9-87f5-4154-9ee1-3addc84e73ec" containerName="payload-processing"
Sep 04 23:12:28.641860 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.641847    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy"
Sep 04 23:12:28.641860 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.641857    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy"
Sep 04 23:12:28.642033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.641879    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3519936-dc95-4dd3-8b08-b0d629fa829e" containerName="revoke-keys"
Sep 04 23:12:28.642033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.641888    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3519936-dc95-4dd3-8b08-b0d629fa829e" containerName="revoke-keys"
Sep 04 23:12:28.642033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.641986    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy"
Sep 04 23:12:28.642033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.642002    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d009f8d9-87f5-4154-9ee1-3addc84e73ec" containerName="payload-processing"
Sep 04 23:12:28.642033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.642013    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3519936-dc95-4dd3-8b08-b0d629fa829e" containerName="revoke-keys"
Sep 04 23:12:28.646238 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.646220    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f"
Sep 04 23:12:28.649243 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.649213    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:12:28.652393 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.652371    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f"]
Sep 04 23:12:28.734270 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.734238    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:28.750437 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.750405    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6d3e24-4aaa-4083-857c-b3b84f80e44f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f\" (UID: \"9f6d3e24-4aaa-4083-857c-b3b84f80e44f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f"
Sep 04 23:12:28.851345 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.851303    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6d3e24-4aaa-4083-857c-b3b84f80e44f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f\" (UID: \"9f6d3e24-4aaa-4083-857c-b3b84f80e44f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f"
Sep 04 23:12:28.851956 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.851937    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6d3e24-4aaa-4083-857c-b3b84f80e44f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f\" (UID: \"9f6d3e24-4aaa-4083-857c-b3b84f80e44f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f"
Sep 04 23:12:28.959488 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:28.959407    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f"
Sep 04 23:12:29.229577 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:29.229554    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f"]
Sep 04 23:12:29.231110 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:12:29.231085    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6d3e24_4aaa_4083_857c_b3b84f80e44f.slice/crio-5843222b84b880175581c6a9622d0481b25a5a15a33706e2298a7a1a740c8003 WatchSource:0}: Error finding container 5843222b84b880175581c6a9622d0481b25a5a15a33706e2298a7a1a740c8003: Status 404 returned error can't find the container with id 5843222b84b880175581c6a9622d0481b25a5a15a33706e2298a7a1a740c8003
Sep 04 23:12:29.643555 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:29.643518    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f" event={"ID":"9f6d3e24-4aaa-4083-857c-b3b84f80e44f","Type":"ContainerStarted","Data":"0f687e177d18deaa76b51d9ffd24f1a86975a9a5d78ee436100b9a837a0b61a0"}
Sep 04 23:12:29.643936 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:29.643570    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f" event={"ID":"9f6d3e24-4aaa-4083-857c-b3b84f80e44f","Type":"ContainerStarted","Data":"5843222b84b880175581c6a9622d0481b25a5a15a33706e2298a7a1a740c8003"}
Sep 04 23:12:29.659013 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:29.658960    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f" podStartSLOduration=1.658943694 podStartE2EDuration="1.658943694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:12:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:12:29.657917205 +0000 UTC m=+1637.768037792" watchObservedRunningTime="2026-09-04 23:12:29.658943694 +0000 UTC m=+1637.769064281"
Sep 04 23:12:30.651574 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:30.651533    2578 generic.go:356] "Generic (PLEG): container finished" podID="9f6d3e24-4aaa-4083-857c-b3b84f80e44f" containerID="0f687e177d18deaa76b51d9ffd24f1a86975a9a5d78ee436100b9a837a0b61a0" exitCode=0
Sep 04 23:12:30.652055 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:30.651595    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f" event={"ID":"9f6d3e24-4aaa-4083-857c-b3b84f80e44f","Type":"ContainerDied","Data":"0f687e177d18deaa76b51d9ffd24f1a86975a9a5d78ee436100b9a837a0b61a0"}
Sep 04 23:12:31.793246 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:31.793221    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f"
Sep 04 23:12:31.882224 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:31.882185    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6d3e24-4aaa-4083-857c-b3b84f80e44f-maas-api-service-ca\") pod \"9f6d3e24-4aaa-4083-857c-b3b84f80e44f\" (UID: \"9f6d3e24-4aaa-4083-857c-b3b84f80e44f\") "
Sep 04 23:12:31.882553 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:31.882528    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f6d3e24-4aaa-4083-857c-b3b84f80e44f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9f6d3e24-4aaa-4083-857c-b3b84f80e44f" (UID: "9f6d3e24-4aaa-4083-857c-b3b84f80e44f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:12:31.983903 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:31.983821    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6d3e24-4aaa-4083-857c-b3b84f80e44f-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:32.665183 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:32.665153    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f"
Sep 04 23:12:32.665360 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:32.665227    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f" event={"ID":"9f6d3e24-4aaa-4083-857c-b3b84f80e44f","Type":"ContainerDied","Data":"5843222b84b880175581c6a9622d0481b25a5a15a33706e2298a7a1a740c8003"}
Sep 04 23:12:32.665360 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:32.665299    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5843222b84b880175581c6a9622d0481b25a5a15a33706e2298a7a1a740c8003"
Sep 04 23:12:33.517917 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.517881    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:33.523529 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.523507    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:12:33.600160 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600122    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istiod-ca-cert\") pod \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") "
Sep 04 23:12:33.600360 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600165    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-credential-socket\") pod \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") "
Sep 04 23:12:33.600360 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600204    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-data\") pod \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") "
Sep 04 23:12:33.600360 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600241    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-workload-socket\") pod \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") "
Sep 04 23:12:33.600360 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600330    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-token\") pod \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") "
Sep 04 23:12:33.600577 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600361    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2289m\" (UniqueName: \"kubernetes.io/projected/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-kube-api-access-2289m\") pod \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") "
Sep 04 23:12:33.600577 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600406    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-workload-certs\") pod \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") "
Sep 04 23:12:33.600577 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600444    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-podinfo\") pod \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") "
Sep 04 23:12:33.600577 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600455    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-data" (OuterVolumeSpecName: "istio-data") pod "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" (UID: "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:33.600577 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600505    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-envoy\") pod \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\" (UID: \"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4\") "
Sep 04 23:12:33.600577 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600560    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" (UID: "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:12:33.600902 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600814    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" (UID: "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:33.600902 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600866    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:33.600902 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.600885    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:33.601231 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.601182    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" (UID: "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:33.601491 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.601364    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" (UID: "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:33.604221 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.604176    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" (UID: "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:12:33.604979 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.604457    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" (UID: "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:33.605367 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.605347    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" (UID: "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:12:33.605549 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.605520    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-kube-api-access-2289m" (OuterVolumeSpecName: "kube-api-access-2289m") pod "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" (UID: "ff08f0ae-d44d-424b-92a4-1ce69d2b17f4"). InnerVolumeSpecName "kube-api-access-2289m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:12:33.672815 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.672781    2578 generic.go:356] "Generic (PLEG): container finished" podID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerID="fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f" exitCode=0
Sep 04 23:12:33.672963 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.672836    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"
Sep 04 23:12:33.672963 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.672863    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" event={"ID":"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4","Type":"ContainerDied","Data":"fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f"}
Sep 04 23:12:33.672963 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.672921    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn" event={"ID":"ff08f0ae-d44d-424b-92a4-1ce69d2b17f4","Type":"ContainerDied","Data":"18fb0f0aeef6b2a5f2704612c7ab0d00a20b0159cce520770d5df5991646312c"}
Sep 04 23:12:33.672963 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.672951    2578 scope.go:117] "RemoveContainer" containerID="fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f"
Sep 04 23:12:33.683122 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.683106    2578 scope.go:117] "RemoveContainer" containerID="fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e"
Sep 04 23:12:33.692207 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.692172    2578 scope.go:117] "RemoveContainer" containerID="fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f"
Sep 04 23:12:33.692493 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:12:33.692474    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f\": container with ID starting with fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f not found: ID does not exist" containerID="fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f"
Sep 04 23:12:33.692556 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.692501    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f"} err="failed to get container status \"fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f\": rpc error: code = NotFound desc = could not find container \"fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f\": container with ID starting with fc2d9e45ebb165ae0f684ab79d137ec79eb0edfaa9d8080967f23e5bb2c99b1f not found: ID does not exist"
Sep 04 23:12:33.692556 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.692517    2578 scope.go:117] "RemoveContainer" containerID="fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e"
Sep 04 23:12:33.692832 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:12:33.692815    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e\": container with ID starting with fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e not found: ID does not exist" containerID="fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e"
Sep 04 23:12:33.692891 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.692837    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e"} err="failed to get container status \"fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e\": rpc error: code = NotFound desc = could not find container \"fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e\": container with ID starting with fd4a84c14a207a98bac0d7731bbac4a97e767e3bec23b3ffbb223cc21b93fb4e not found: ID does not exist"
Sep 04 23:12:33.697520 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.697485    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"]
Sep 04 23:12:33.703284 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.703261    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:33.703284 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.703286    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2289m\" (UniqueName: \"kubernetes.io/projected/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-kube-api-access-2289m\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:33.703414 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.703296    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:33.703414 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.703305    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:33.703414 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.703313    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:33.703414 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.703321    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:33.703414 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.703328    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:33.703562 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:33.703465    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9b6db4-openshift-default-5d94699676-bttdn"]
Sep 04 23:12:34.281914 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.281878    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:34.419964 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.419930    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" path="/var/lib/kubelet/pods/ff08f0ae-d44d-424b-92a4-1ce69d2b17f4/volumes"
Sep 04 23:12:34.637523 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.637489    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:34.907581 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.907499    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"]
Sep 04 23:12:34.908019 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.908000    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f6d3e24-4aaa-4083-857c-b3b84f80e44f" containerName="revoke-keys"
Sep 04 23:12:34.908113 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.908022    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f6d3e24-4aaa-4083-857c-b3b84f80e44f" containerName="revoke-keys"
Sep 04 23:12:34.908113 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.908048    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerName="istio-proxy"
Sep 04 23:12:34.908113 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.908057    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerName="istio-proxy"
Sep 04 23:12:34.908113 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.908077    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerName="istio-proxy"
Sep 04 23:12:34.908113 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.908085    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerName="istio-proxy"
Sep 04 23:12:34.908365 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.908208    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerName="istio-proxy"
Sep 04 23:12:34.908365 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.908222    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff08f0ae-d44d-424b-92a4-1ce69d2b17f4" containerName="istio-proxy"
Sep 04 23:12:34.908365 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.908231    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f6d3e24-4aaa-4083-857c-b3b84f80e44f" containerName="revoke-keys"
Sep 04 23:12:34.908365 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.908245    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f577ede7-e488-4160-ab2d-ee551c7f1a97" containerName="istio-proxy"
Sep 04 23:12:34.913057 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.913033    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:34.915408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.915389    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-3543cf-openshift-default-dockercfg-txpmg\""
Sep 04 23:12:34.922502 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.922479    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"]
Sep 04 23:12:34.937376 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:34.937350    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:35.016463 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.016427    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-token\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.016627 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.016471    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-credential-socket\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.016627 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.016523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql8k7\" (UniqueName: \"kubernetes.io/projected/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-kube-api-access-ql8k7\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.016627 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.016558    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-podinfo\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.016627 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.016603    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-envoy\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.016850 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.016669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-workload-socket\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.016850 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.016706    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-workload-certs\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.016850 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.016730    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istiod-ca-cert\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.016978 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.016860    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-data\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.034581 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.034553    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:35.118495 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.118455    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-data\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.118735 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.118548    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-token\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.118735 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.118581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-credential-socket\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.118735 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.118612    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql8k7\" (UniqueName: \"kubernetes.io/projected/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-kube-api-access-ql8k7\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.118735 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.118657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-podinfo\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.118735 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.118689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-envoy\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.118735 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.118732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-workload-socket\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.119107 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.118787    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-workload-certs\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.119107 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.118913    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-data\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.119107 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.119038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-credential-socket\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.119107 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.119081    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-workload-socket\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.119317 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.119207    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-workload-certs\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.119317 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.119250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istiod-ca-cert\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.119536 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.119518    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istiod-ca-cert\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.121069 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.121038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-envoy\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.121339 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.121318    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-podinfo\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.126471 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.126452    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-token\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.126614 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.126595    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql8k7\" (UniqueName: \"kubernetes.io/projected/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-kube-api-access-ql8k7\") pod \"e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") " pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.225171 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.225078    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:35.510822 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.510793    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"]
Sep 04 23:12:35.512807 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:12:35.512778    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5e6d6e_7b61_4744_b9a7_ca7970553fcf.slice/crio-d62a47d02041d6fa02203625817ed9fe19529c38835a2fd626a8a7dfc24254e2 WatchSource:0}: Error finding container d62a47d02041d6fa02203625817ed9fe19529c38835a2fd626a8a7dfc24254e2: Status 404 returned error can't find the container with id d62a47d02041d6fa02203625817ed9fe19529c38835a2fd626a8a7dfc24254e2
Sep 04 23:12:35.515304 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.515266    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:12:35.515400 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.515368    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:12:35.515436 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.515415    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:12:35.691609 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.691573    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" event={"ID":"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf","Type":"ContainerStarted","Data":"d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44"}
Sep 04 23:12:35.692021 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.691632    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" event={"ID":"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf","Type":"ContainerStarted","Data":"d62a47d02041d6fa02203625817ed9fe19529c38835a2fd626a8a7dfc24254e2"}
Sep 04 23:12:35.712578 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:35.712520    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" podStartSLOduration=1.712504194 podStartE2EDuration="1.712504194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:12:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:12:35.709179992 +0000 UTC m=+1643.819300578" watchObservedRunningTime="2026-09-04 23:12:35.712504194 +0000 UTC m=+1643.822624779"
Sep 04 23:12:36.226118 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:36.226083    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:36.231913 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:36.231884    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:36.697695 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:36.697667    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:36.698628 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:36.698605    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:37.604185 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:37.604142    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:37.698635 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:37.698594    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:38.760380 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:38.760343    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:38.863190 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:38.863159    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:39.341961 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.341921    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:39.439741 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.439712    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:39.485542 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.485504    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"]
Sep 04 23:12:39.489413 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.489397    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"
Sep 04 23:12:39.492121 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.492099    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-3543cf\""
Sep 04 23:12:39.492233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.492154    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-3543cf-dockercfg-67nzm\""
Sep 04 23:12:39.497495 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.497472    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"]
Sep 04 23:12:39.654352 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.654279    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:39.666728 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.666693    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc6d19fa-ad4c-4828-b804-3e6208ed17a1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f\" (UID: \"dc6d19fa-ad4c-4828-b804-3e6208ed17a1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"
Sep 04 23:12:39.666943 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.666909    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2rkj\" (UniqueName: \"kubernetes.io/projected/dc6d19fa-ad4c-4828-b804-3e6208ed17a1-kube-api-access-c2rkj\") pod \"payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f\" (UID: \"dc6d19fa-ad4c-4828-b804-3e6208ed17a1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"
Sep 04 23:12:39.696425 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.696390    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"]
Sep 04 23:12:39.696649 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.696609    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb" podUID="855ef520-6260-427b-abea-412e97ea30ec" containerName="payload-pre-processing" containerID="cri-o://6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9" gracePeriod=30
Sep 04 23:12:39.754920 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.754866    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:39.768185 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.768150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc6d19fa-ad4c-4828-b804-3e6208ed17a1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f\" (UID: \"dc6d19fa-ad4c-4828-b804-3e6208ed17a1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"
Sep 04 23:12:39.768511 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.768201    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2rkj\" (UniqueName: \"kubernetes.io/projected/dc6d19fa-ad4c-4828-b804-3e6208ed17a1-kube-api-access-c2rkj\") pod \"payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f\" (UID: \"dc6d19fa-ad4c-4828-b804-3e6208ed17a1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"
Sep 04 23:12:39.768858 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.768836    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc6d19fa-ad4c-4828-b804-3e6208ed17a1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f\" (UID: \"dc6d19fa-ad4c-4828-b804-3e6208ed17a1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"
Sep 04 23:12:39.777323 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.777298    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2rkj\" (UniqueName: \"kubernetes.io/projected/dc6d19fa-ad4c-4828-b804-3e6208ed17a1-kube-api-access-c2rkj\") pod \"payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f\" (UID: \"dc6d19fa-ad4c-4828-b804-3e6208ed17a1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"
Sep 04 23:12:39.802377 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.802350    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"
Sep 04 23:12:39.856279 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.856138    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:39.918238 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:39.918116    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:40.019020 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.018991    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"
Sep 04 23:12:40.023886 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.023853    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:40.095757 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.095717    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f"]
Sep 04 23:12:40.102616 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.102587    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ed43d3-7ql5f"]
Sep 04 23:12:40.120058 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.120036    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"]
Sep 04 23:12:40.121425 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:12:40.121398    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc6d19fa_ad4c_4828_b804_3e6208ed17a1.slice/crio-c9ae147b556f663227cc0afbb25e467d7d571ca6ed027864ca6efb329d3fc5e6 WatchSource:0}: Error finding container c9ae147b556f663227cc0afbb25e467d7d571ca6ed027864ca6efb329d3fc5e6: Status 404 returned error can't find the container with id c9ae147b556f663227cc0afbb25e467d7d571ca6ed027864ca6efb329d3fc5e6
Sep 04 23:12:40.179461 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.179438    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855ef520-6260-427b-abea-412e97ea30ec-plugins-config-volume\") pod \"855ef520-6260-427b-abea-412e97ea30ec\" (UID: \"855ef520-6260-427b-abea-412e97ea30ec\") "
Sep 04 23:12:40.179575 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.179518    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkt5f\" (UniqueName: \"kubernetes.io/projected/855ef520-6260-427b-abea-412e97ea30ec-kube-api-access-qkt5f\") pod \"855ef520-6260-427b-abea-412e97ea30ec\" (UID: \"855ef520-6260-427b-abea-412e97ea30ec\") "
Sep 04 23:12:40.179849 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.179825    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/855ef520-6260-427b-abea-412e97ea30ec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "855ef520-6260-427b-abea-412e97ea30ec" (UID: "855ef520-6260-427b-abea-412e97ea30ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:12:40.181502 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.181474    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/855ef520-6260-427b-abea-412e97ea30ec-kube-api-access-qkt5f" (OuterVolumeSpecName: "kube-api-access-qkt5f") pod "855ef520-6260-427b-abea-412e97ea30ec" (UID: "855ef520-6260-427b-abea-412e97ea30ec"). InnerVolumeSpecName "kube-api-access-qkt5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:12:40.281026 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.280984    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855ef520-6260-427b-abea-412e97ea30ec-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:40.281026 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.281018    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkt5f\" (UniqueName: \"kubernetes.io/projected/855ef520-6260-427b-abea-412e97ea30ec-kube-api-access-qkt5f\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:40.423048 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.423000    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f6d3e24-4aaa-4083-857c-b3b84f80e44f" path="/var/lib/kubelet/pods/9f6d3e24-4aaa-4083-857c-b3b84f80e44f/volumes"
Sep 04 23:12:40.452160 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.452131    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:40.730240 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.730206    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f" event={"ID":"dc6d19fa-ad4c-4828-b804-3e6208ed17a1","Type":"ContainerStarted","Data":"06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2"}
Sep 04 23:12:40.730397 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.730272    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f" event={"ID":"dc6d19fa-ad4c-4828-b804-3e6208ed17a1","Type":"ContainerStarted","Data":"c9ae147b556f663227cc0afbb25e467d7d571ca6ed027864ca6efb329d3fc5e6"}
Sep 04 23:12:40.730397 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.730304    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"
Sep 04 23:12:40.731561 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.731535    2578 generic.go:356] "Generic (PLEG): container finished" podID="855ef520-6260-427b-abea-412e97ea30ec" containerID="6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9" exitCode=0
Sep 04 23:12:40.731694 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.731567    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"
Sep 04 23:12:40.731694 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.731601    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb" event={"ID":"855ef520-6260-427b-abea-412e97ea30ec","Type":"ContainerDied","Data":"6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9"}
Sep 04 23:12:40.731694 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.731635    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb" event={"ID":"855ef520-6260-427b-abea-412e97ea30ec","Type":"ContainerDied","Data":"3b6bbf5f12981b4374fce91424bbd6d7a42cb35966075da01f1d74083d81ff5d"}
Sep 04 23:12:40.731694 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.731671    2578 scope.go:117] "RemoveContainer" containerID="6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9"
Sep 04 23:12:40.741547 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.741530    2578 scope.go:117] "RemoveContainer" containerID="6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9"
Sep 04 23:12:40.741829 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:12:40.741806    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9\": container with ID starting with 6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9 not found: ID does not exist" containerID="6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9"
Sep 04 23:12:40.741892 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.741840    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9"} err="failed to get container status \"6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9\": rpc error: code = NotFound desc = could not find container \"6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9\": container with ID starting with 6432bedce9940ff890e0e986c4815c3ae7fb1a266fe21e9d2e0b2dcfaee455a9 not found: ID does not exist"
Sep 04 23:12:40.748775 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.748736    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f" podStartSLOduration=1.7487232769999999 podStartE2EDuration="1.748723277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:12:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:12:40.745711911 +0000 UTC m=+1648.855832502" watchObservedRunningTime="2026-09-04 23:12:40.748723277 +0000 UTC m=+1648.858843862"
Sep 04 23:12:40.762348 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.762323    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"]
Sep 04 23:12:40.764863 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:40.764841    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb"]
Sep 04 23:12:41.005179 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:41.005075    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ed43d3-6db947b9d6-jkpvb" podUID="855ef520-6260-427b-abea-412e97ea30ec" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.75:9004: i/o timeout"
Sep 04 23:12:42.419794 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:42.419761    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="855ef520-6260-427b-abea-412e97ea30ec" path="/var/lib/kubelet/pods/855ef520-6260-427b-abea-412e97ea30ec/volumes"
Sep 04 23:12:45.536899 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:45.536861    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:46.709472 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:46.709401    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:46.812366 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:46.812322    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:46.965751 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:46.965233    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"]
Sep 04 23:12:46.965751 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:46.965482    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy" containerID="cri-o://4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0" gracePeriod=30
Sep 04 23:12:46.967081 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:46.967048    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:47.590309 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:47.590273    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:49.679138 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:49.679099    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 04 23:12:49.679495 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:49.679179    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 04 23:12:51.742071 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:51.742040    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"
Sep 04 23:12:52.219766 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.219738    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:12:52.313347 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.313309    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zj7k\" (UniqueName: \"kubernetes.io/projected/342cede2-d66e-4584-a747-ba5abbf72b10-kube-api-access-6zj7k\") pod \"342cede2-d66e-4584-a747-ba5abbf72b10\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") "
Sep 04 23:12:52.313663 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.313629    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/342cede2-d66e-4584-a747-ba5abbf72b10-istio-token\") pod \"342cede2-d66e-4584-a747-ba5abbf72b10\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") "
Sep 04 23:12:52.313798 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.313785    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-workload-certs\") pod \"342cede2-d66e-4584-a747-ba5abbf72b10\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") "
Sep 04 23:12:52.313951 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.313938    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-istio-data\") pod \"342cede2-d66e-4584-a747-ba5abbf72b10\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") "
Sep 04 23:12:52.314165 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.314142    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/342cede2-d66e-4584-a747-ba5abbf72b10-istio-podinfo\") pod \"342cede2-d66e-4584-a747-ba5abbf72b10\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") "
Sep 04 23:12:52.314450 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.314115    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "342cede2-d66e-4584-a747-ba5abbf72b10" (UID: "342cede2-d66e-4584-a747-ba5abbf72b10"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:52.314450 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.314199    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-istio-data" (OuterVolumeSpecName: "istio-data") pod "342cede2-d66e-4584-a747-ba5abbf72b10" (UID: "342cede2-d66e-4584-a747-ba5abbf72b10"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:52.314450 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.314385    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-credential-socket\") pod \"342cede2-d66e-4584-a747-ba5abbf72b10\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") "
Sep 04 23:12:52.314875 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.314837    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "342cede2-d66e-4584-a747-ba5abbf72b10" (UID: "342cede2-d66e-4584-a747-ba5abbf72b10"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:52.314995 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.314970    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/342cede2-d66e-4584-a747-ba5abbf72b10-istiod-ca-cert\") pod \"342cede2-d66e-4584-a747-ba5abbf72b10\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") "
Sep 04 23:12:52.315465 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.315440    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/342cede2-d66e-4584-a747-ba5abbf72b10-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "342cede2-d66e-4584-a747-ba5abbf72b10" (UID: "342cede2-d66e-4584-a747-ba5abbf72b10"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:12:52.315588 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.315572    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-istio-envoy\") pod \"342cede2-d66e-4584-a747-ba5abbf72b10\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") "
Sep 04 23:12:52.316288 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.316249    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "342cede2-d66e-4584-a747-ba5abbf72b10" (UID: "342cede2-d66e-4584-a747-ba5abbf72b10"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:52.316838 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.316801    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/342cede2-d66e-4584-a747-ba5abbf72b10-istio-token" (OuterVolumeSpecName: "istio-token") pod "342cede2-d66e-4584-a747-ba5abbf72b10" (UID: "342cede2-d66e-4584-a747-ba5abbf72b10"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:12:52.317567 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.317502    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/342cede2-d66e-4584-a747-ba5abbf72b10-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "342cede2-d66e-4584-a747-ba5abbf72b10" (UID: "342cede2-d66e-4584-a747-ba5abbf72b10"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:12:52.317699 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.317676    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/342cede2-d66e-4584-a747-ba5abbf72b10-kube-api-access-6zj7k" (OuterVolumeSpecName: "kube-api-access-6zj7k") pod "342cede2-d66e-4584-a747-ba5abbf72b10" (UID: "342cede2-d66e-4584-a747-ba5abbf72b10"). InnerVolumeSpecName "kube-api-access-6zj7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:12:52.317992 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.316036    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-workload-socket\") pod \"342cede2-d66e-4584-a747-ba5abbf72b10\" (UID: \"342cede2-d66e-4584-a747-ba5abbf72b10\") "
Sep 04 23:12:52.318339 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.318318    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:52.318429 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.318341    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/342cede2-d66e-4584-a747-ba5abbf72b10-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:52.318429 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.318357    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:52.318429 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.318370    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/342cede2-d66e-4584-a747-ba5abbf72b10-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:52.318429 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.318384    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:52.318429 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.318397    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zj7k\" (UniqueName: \"kubernetes.io/projected/342cede2-d66e-4584-a747-ba5abbf72b10-kube-api-access-6zj7k\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:52.318429 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.318410    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/342cede2-d66e-4584-a747-ba5abbf72b10-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:52.318429 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.318424    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:52.319280 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.319256    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "342cede2-d66e-4584-a747-ba5abbf72b10" (UID: "342cede2-d66e-4584-a747-ba5abbf72b10"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:12:52.419025 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.418953    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/342cede2-d66e-4584-a747-ba5abbf72b10-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:12:52.822602 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.822515    2578 generic.go:356] "Generic (PLEG): container finished" podID="342cede2-d66e-4584-a747-ba5abbf72b10" containerID="4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0" exitCode=0
Sep 04 23:12:52.822602 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.822577    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"
Sep 04 23:12:52.823092 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.822611    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" event={"ID":"342cede2-d66e-4584-a747-ba5abbf72b10","Type":"ContainerDied","Data":"4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0"}
Sep 04 23:12:52.823092 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.822691    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg" event={"ID":"342cede2-d66e-4584-a747-ba5abbf72b10","Type":"ContainerDied","Data":"fc512ea5a172e8d91454d24f632b8adb84fdfe197cd01b472d2d2b771434350f"}
Sep 04 23:12:52.823092 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.822720    2578 scope.go:117] "RemoveContainer" containerID="4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0"
Sep 04 23:12:52.833544 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.833521    2578 scope.go:117] "RemoveContainer" containerID="435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb"
Sep 04 23:12:52.841322 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.841295    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"]
Sep 04 23:12:52.844456 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.844436    2578 scope.go:117] "RemoveContainer" containerID="4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0"
Sep 04 23:12:52.844812 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:12:52.844790    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0\": container with ID starting with 4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0 not found: ID does not exist" containerID="4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0"
Sep 04 23:12:52.844901 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.844820    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0"} err="failed to get container status \"4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0\": rpc error: code = NotFound desc = could not find container \"4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0\": container with ID starting with 4cea93850c2fb96cbe95c84938db36db7c6e3ac4300ac68c941f9e56975ad0b0 not found: ID does not exist"
Sep 04 23:12:52.844901 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.844840    2578 scope.go:117] "RemoveContainer" containerID="435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb"
Sep 04 23:12:52.845142 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:12:52.845125    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb\": container with ID starting with 435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb not found: ID does not exist" containerID="435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb"
Sep 04 23:12:52.845206 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.845150    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb"} err="failed to get container status \"435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb\": rpc error: code = NotFound desc = could not find container \"435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb\": container with ID starting with 435ac8c3c08a33fc1ef364e90788a30e5a2c5cc048fdda1b8430562038970beb not found: ID does not exist"
Sep 04 23:12:52.845206 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:52.845133    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ed43d3-openshift-default-84765cbc68-ljvxg"]
Sep 04 23:12:54.427511 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:54.427476    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" path="/var/lib/kubelet/pods/342cede2-d66e-4584-a747-ba5abbf72b10/volumes"
Sep 04 23:12:55.846532 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:55.846496    2578 generic.go:356] "Generic (PLEG): container finished" podID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerID="d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44" exitCode=0
Sep 04 23:12:55.847005 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:55.846569    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" event={"ID":"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf","Type":"ContainerDied","Data":"d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44"}
Sep 04 23:12:55.847005 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:55.846990    2578 scope.go:117] "RemoveContainer" containerID="d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44"
Sep 04 23:12:55.848187 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:55.848153    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:12:55.848268 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:55.848237    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:12:55.848325 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:55.848273    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:12:56.861295 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:56.861255    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" event={"ID":"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf","Type":"ContainerStarted","Data":"aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512"}
Sep 04 23:12:57.225517 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:57.225429    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:12:57.225688 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:57.225635    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 04 23:12:57.225744 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:57.225722    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 04 23:12:57.805257 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:57.805223    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:57.906825 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:57.906791    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:58.007006 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:58.006974    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:58.064710 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:58.064600    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:58.226179 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:58.226145    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 04 23:12:58.226344 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:58.226200    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 04 23:12:58.391283 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:58.391249    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:12:59.225527 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:59.225484    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 04 23:12:59.225939 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:59.225559    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 04 23:12:59.733814 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:12:59.733768    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:00.225861 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:00.225821    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 04 23:13:00.226242 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:00.225891    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 04 23:13:01.225480 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:01.225439    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 04 23:13:01.225693 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:01.225498    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 04 23:13:02.229736 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:02.229702    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:13:02.230205 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:02.230090    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:13:02.230810 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:02.230791    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:13:02.598110 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:02.598080    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:03.248354 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:03.248310    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:03.557249 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:03.557173    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:03.846915 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:03.846882    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:04.582871 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:04.582839    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:04.683835 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:04.683787    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:04.834619 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:04.834517    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"]
Sep 04 23:13:04.834870 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:04.834834    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy" containerID="cri-o://09ea264f6ab1f1bc7eb67afd751cd24fa7d17594b966e0ba68ac270ec542f97f" gracePeriod=30
Sep 04 23:13:04.855315 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:04.855270    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:06.338347 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:06.338299    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:06.438251 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:06.438201    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:07.638456 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:07.638408    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:07.737389 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:07.737355    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:07.841402 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:07.841361    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:07.906201 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:07.906118    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 04 23:13:07.906356 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:07.906192    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 04 23:13:08.106457 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:08.106428    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:08.195667 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:08.195567    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:09.958505 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:09.958470    2578 generic.go:356] "Generic (PLEG): container finished" podID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerID="09ea264f6ab1f1bc7eb67afd751cd24fa7d17594b966e0ba68ac270ec542f97f" exitCode=0
Sep 04 23:13:09.958895 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:09.958542    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" event={"ID":"1262d309-5b05-42aa-ba2c-5c0f0f703dae","Type":"ContainerDied","Data":"09ea264f6ab1f1bc7eb67afd751cd24fa7d17594b966e0ba68ac270ec542f97f"}
Sep 04 23:13:09.958895 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:09.958668    2578 scope.go:117] "RemoveContainer" containerID="49a0b4d03729ed94686ae5d1b52b7cb5d94938f15b666431e52b112605901a4b"
Sep 04 23:13:10.093522 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.093497    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:13:10.197625 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.197523    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-workload-certs\") pod \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") "
Sep 04 23:13:10.197625 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.197580    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-credential-socket\") pod \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") "
Sep 04 23:13:10.197625 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.197619    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-podinfo\") pod \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") "
Sep 04 23:13:10.197958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.197662    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-workload-socket\") pod \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") "
Sep 04 23:13:10.197958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.197698    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-envoy\") pod \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") "
Sep 04 23:13:10.197958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.197720    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbw45\" (UniqueName: \"kubernetes.io/projected/1262d309-5b05-42aa-ba2c-5c0f0f703dae-kube-api-access-rbw45\") pod \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") "
Sep 04 23:13:10.197958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.197773    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-data\") pod \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") "
Sep 04 23:13:10.197958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.197816    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istiod-ca-cert\") pod \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") "
Sep 04 23:13:10.197958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.197892    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1262d309-5b05-42aa-ba2c-5c0f0f703dae" (UID: "1262d309-5b05-42aa-ba2c-5c0f0f703dae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:13:10.197958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.197857    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-token\") pod \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\" (UID: \"1262d309-5b05-42aa-ba2c-5c0f0f703dae\") "
Sep 04 23:13:10.198268 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.197980    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1262d309-5b05-42aa-ba2c-5c0f0f703dae" (UID: "1262d309-5b05-42aa-ba2c-5c0f0f703dae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:13:10.198268 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.197994    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1262d309-5b05-42aa-ba2c-5c0f0f703dae" (UID: "1262d309-5b05-42aa-ba2c-5c0f0f703dae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:13:10.198268 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.198073    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-data" (OuterVolumeSpecName: "istio-data") pod "1262d309-5b05-42aa-ba2c-5c0f0f703dae" (UID: "1262d309-5b05-42aa-ba2c-5c0f0f703dae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:13:10.198421 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.198333    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1262d309-5b05-42aa-ba2c-5c0f0f703dae" (UID: "1262d309-5b05-42aa-ba2c-5c0f0f703dae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:13:10.198421 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.198360    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:13:10.198421 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.198380    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:13:10.198421 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.198395    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:13:10.198421 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.198410    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:13:10.199888 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.199859    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1262d309-5b05-42aa-ba2c-5c0f0f703dae" (UID: "1262d309-5b05-42aa-ba2c-5c0f0f703dae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:13:10.200203 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.200182    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1262d309-5b05-42aa-ba2c-5c0f0f703dae-kube-api-access-rbw45" (OuterVolumeSpecName: "kube-api-access-rbw45") pod "1262d309-5b05-42aa-ba2c-5c0f0f703dae" (UID: "1262d309-5b05-42aa-ba2c-5c0f0f703dae"). InnerVolumeSpecName "kube-api-access-rbw45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:13:10.200389 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.200372    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1262d309-5b05-42aa-ba2c-5c0f0f703dae" (UID: "1262d309-5b05-42aa-ba2c-5c0f0f703dae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:13:10.200434 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.200403    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-token" (OuterVolumeSpecName: "istio-token") pod "1262d309-5b05-42aa-ba2c-5c0f0f703dae" (UID: "1262d309-5b05-42aa-ba2c-5c0f0f703dae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:13:10.299111 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.299068    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:13:10.299111 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.299109    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:13:10.299111 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.299118    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:13:10.299111 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.299126    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1262d309-5b05-42aa-ba2c-5c0f0f703dae-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:13:10.299381 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.299135    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbw45\" (UniqueName: \"kubernetes.io/projected/1262d309-5b05-42aa-ba2c-5c0f0f703dae-kube-api-access-rbw45\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:13:10.966168 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.966130    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g" event={"ID":"1262d309-5b05-42aa-ba2c-5c0f0f703dae","Type":"ContainerDied","Data":"ece9a984183249ae0e053469145c507b127d51656c6c01f72e81cdac644aebef"}
Sep 04 23:13:10.966635 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.966188    2578 scope.go:117] "RemoveContainer" containerID="09ea264f6ab1f1bc7eb67afd751cd24fa7d17594b966e0ba68ac270ec542f97f"
Sep 04 23:13:10.966635 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.966136    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"
Sep 04 23:13:10.983923 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.983899    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"]
Sep 04 23:13:10.987677 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:10.987655    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-217ad6-openshift-default-79b9799b74-b6k4g"]
Sep 04 23:13:12.420538 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:12.420506    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" path="/var/lib/kubelet/pods/1262d309-5b05-42aa-ba2c-5c0f0f703dae/volumes"
Sep 04 23:13:31.374807 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:31.374763    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:36.416360 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:36.416256    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-2pxrz_337ff73a-57c3-4d64-a0a6-e942e76477d5/payload-processing/0.log"
Sep 04 23:13:50.131196 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:50.131158    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:13:51.926082 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:13:51.926052    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:12.213989 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:12.213959    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f_dc6d19fa-ad4c-4828-b804-3e6208ed17a1/payload-processing/0.log"
Sep 04 23:14:12.338747 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:12.338717    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-2pxrz_337ff73a-57c3-4d64-a0a6-e942e76477d5/payload-processing/0.log"
Sep 04 23:14:12.748489 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:12.748457    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:13.028167 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028069    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"]
Sep 04 23:14:13.028512 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028493    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy"
Sep 04 23:14:13.028512 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028510    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy"
Sep 04 23:14:13.028687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028521    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy"
Sep 04 23:14:13.028687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028527    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy"
Sep 04 23:14:13.028687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028534    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy"
Sep 04 23:14:13.028687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028540    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy"
Sep 04 23:14:13.028687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028552    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855ef520-6260-427b-abea-412e97ea30ec" containerName="payload-pre-processing"
Sep 04 23:14:13.028687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028558    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="855ef520-6260-427b-abea-412e97ea30ec" containerName="payload-pre-processing"
Sep 04 23:14:13.028687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028659    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy"
Sep 04 23:14:13.028687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028667    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="855ef520-6260-427b-abea-412e97ea30ec" containerName="payload-pre-processing"
Sep 04 23:14:13.028687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028677    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy"
Sep 04 23:14:13.028687 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.028683    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1262d309-5b05-42aa-ba2c-5c0f0f703dae" containerName="istio-proxy"
Sep 04 23:14:13.035007 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.034972    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.038185 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.038157    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-23158b-openshift-default-dockercfg-rsf29\""
Sep 04 23:14:13.045854 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.045824    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"]
Sep 04 23:14:13.055077 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.055042    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:13.086460 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.086425    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-workload-certs\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.086460 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.086469    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-data\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.086678 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.086486    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-podinfo\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.086678 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.086541    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkk4j\" (UniqueName: \"kubernetes.io/projected/72a952bf-e2fd-4eab-afe4-49b72725251c-kube-api-access-vkk4j\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.086678 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.086580    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-credential-socket\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.086678 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.086602    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-envoy\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.086825 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.086681    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72a952bf-e2fd-4eab-afe4-49b72725251c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.086825 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.086732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-workload-socket\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.086825 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.086750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-token\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.187854 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.187815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72a952bf-e2fd-4eab-afe4-49b72725251c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188037 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.187906    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-workload-socket\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188037 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.187938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-token\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188037 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.187978    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-workload-certs\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188037 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.188015    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-data\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188037 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.188036    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-podinfo\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.188069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkk4j\" (UniqueName: \"kubernetes.io/projected/72a952bf-e2fd-4eab-afe4-49b72725251c-kube-api-access-vkk4j\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.188098    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-credential-socket\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.188122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-envoy\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188470 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.188385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-workload-socket\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188470 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.188456    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-data\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188577 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.188527    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-workload-certs\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188618 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.188588    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-credential-socket\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.188710 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.188692    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72a952bf-e2fd-4eab-afe4-49b72725251c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.190506 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.190482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-envoy\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.190751 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.190733    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-podinfo\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.194994 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.194971    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-token\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.195254 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.195235    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkk4j\" (UniqueName: \"kubernetes.io/projected/72a952bf-e2fd-4eab-afe4-49b72725251c-kube-api-access-vkk4j\") pod \"e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") " pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.352669 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.352610    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:13.353113 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.353091    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:13.628803 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.628775    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"]
Sep 04 23:14:13.630468 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:14:13.630439    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a952bf_e2fd_4eab_afe4_49b72725251c.slice/crio-9761a5c99e8553b256d6879758ccfdf204668f2e441ce14eed255a5ab1eedc3b WatchSource:0}: Error finding container 9761a5c99e8553b256d6879758ccfdf204668f2e441ce14eed255a5ab1eedc3b: Status 404 returned error can't find the container with id 9761a5c99e8553b256d6879758ccfdf204668f2e441ce14eed255a5ab1eedc3b
Sep 04 23:14:13.633107 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.633060    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:14:13.633224 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.633144    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:14:13.633224 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:13.633191    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:14:14.364102 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:14.364066    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" event={"ID":"72a952bf-e2fd-4eab-afe4-49b72725251c","Type":"ContainerStarted","Data":"cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c"}
Sep 04 23:14:14.364521 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:14.364124    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" event={"ID":"72a952bf-e2fd-4eab-afe4-49b72725251c","Type":"ContainerStarted","Data":"9761a5c99e8553b256d6879758ccfdf204668f2e441ce14eed255a5ab1eedc3b"}
Sep 04 23:14:14.385698 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:14.385631    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" podStartSLOduration=1.385617052 podStartE2EDuration="1.385617052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:14:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:14:14.382275006 +0000 UTC m=+1742.492395592" watchObservedRunningTime="2026-09-04 23:14:14.385617052 +0000 UTC m=+1742.495737638"
Sep 04 23:14:15.352956 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:15.352914    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:15.358798 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:15.358767    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:15.369953 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:15.369927    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:15.370683 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:15.370665    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:16.322027 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:16.321982    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:16.407079 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:16.407048    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:17.529820 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:17.529787    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:17.640196 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:17.640135    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:17.737099 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:17.737065    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:18.060948 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.060917    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:18.154504 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.154458    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:18.198008 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.197980    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"]
Sep 04 23:14:18.198437 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.198423    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy"
Sep 04 23:14:18.198497 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.198439    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy"
Sep 04 23:14:18.198534 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.198499    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="342cede2-d66e-4584-a747-ba5abbf72b10" containerName="istio-proxy"
Sep 04 23:14:18.201626 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.201610    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"
Sep 04 23:14:18.204375 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.204350    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-23158b\""
Sep 04 23:14:18.204510 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.204463    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-23158b-dockercfg-w6cxk\""
Sep 04 23:14:18.213309 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.213273    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"]
Sep 04 23:14:18.237902 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.237868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8qjf\" (UniqueName: \"kubernetes.io/projected/7e4afce8-3b14-419b-8172-d85c2171b1a7-kube-api-access-b8qjf\") pod \"payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt\" (UID: \"7e4afce8-3b14-419b-8172-d85c2171b1a7\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"
Sep 04 23:14:18.238087 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.237927    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4afce8-3b14-419b-8172-d85c2171b1a7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt\" (UID: \"7e4afce8-3b14-419b-8172-d85c2171b1a7\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"
Sep 04 23:14:18.339233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.339200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4afce8-3b14-419b-8172-d85c2171b1a7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt\" (UID: \"7e4afce8-3b14-419b-8172-d85c2171b1a7\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"
Sep 04 23:14:18.339404 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.339295    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8qjf\" (UniqueName: \"kubernetes.io/projected/7e4afce8-3b14-419b-8172-d85c2171b1a7-kube-api-access-b8qjf\") pod \"payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt\" (UID: \"7e4afce8-3b14-419b-8172-d85c2171b1a7\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"
Sep 04 23:14:18.340195 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.340161    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4afce8-3b14-419b-8172-d85c2171b1a7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt\" (UID: \"7e4afce8-3b14-419b-8172-d85c2171b1a7\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"
Sep 04 23:14:18.346861 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.346837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8qjf\" (UniqueName: \"kubernetes.io/projected/7e4afce8-3b14-419b-8172-d85c2171b1a7-kube-api-access-b8qjf\") pod \"payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt\" (UID: \"7e4afce8-3b14-419b-8172-d85c2171b1a7\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"
Sep 04 23:14:18.514182 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.514151    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"
Sep 04 23:14:18.792488 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:18.792464    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"]
Sep 04 23:14:18.793383 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:14:18.793357    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4afce8_3b14_419b_8172_d85c2171b1a7.slice/crio-cc7b54a5be9c3376fd7376c525b38dea7a10c3d6f2abf5e47e530b8faf7026b7 WatchSource:0}: Error finding container cc7b54a5be9c3376fd7376c525b38dea7a10c3d6f2abf5e47e530b8faf7026b7: Status 404 returned error can't find the container with id cc7b54a5be9c3376fd7376c525b38dea7a10c3d6f2abf5e47e530b8faf7026b7
Sep 04 23:14:19.397038 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:19.396977    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt" event={"ID":"7e4afce8-3b14-419b-8172-d85c2171b1a7","Type":"ContainerStarted","Data":"a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf"}
Sep 04 23:14:19.397038 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:19.397039    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt" event={"ID":"7e4afce8-3b14-419b-8172-d85c2171b1a7","Type":"ContainerStarted","Data":"cc7b54a5be9c3376fd7376c525b38dea7a10c3d6f2abf5e47e530b8faf7026b7"}
Sep 04 23:14:19.397282 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:19.397067    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"
Sep 04 23:14:19.414126 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:19.414071    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt" podStartSLOduration=1.414053137 podStartE2EDuration="1.414053137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:14:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:14:19.413249113 +0000 UTC m=+1747.523369705" watchObservedRunningTime="2026-09-04 23:14:19.414053137 +0000 UTC m=+1747.524173724"
Sep 04 23:14:30.405080 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:30.405040    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"
Sep 04 23:14:35.510373 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:35.510342    2578 generic.go:356] "Generic (PLEG): container finished" podID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerID="cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c" exitCode=0
Sep 04 23:14:35.510819 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:35.510415    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" event={"ID":"72a952bf-e2fd-4eab-afe4-49b72725251c","Type":"ContainerDied","Data":"cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c"}
Sep 04 23:14:35.510878 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:35.510845    2578 scope.go:117] "RemoveContainer" containerID="cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c"
Sep 04 23:14:35.511994 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:35.511960    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:14:35.512078 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:35.512035    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:14:35.512078 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:35.512066    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 04 23:14:36.518163 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:36.518126    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" event={"ID":"72a952bf-e2fd-4eab-afe4-49b72725251c","Type":"ContainerStarted","Data":"2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760"}
Sep 04 23:14:37.353954 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:37.353915    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:37.354377 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:37.354272    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 04 23:14:37.354553 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:37.354523    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 04 23:14:38.353339 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:38.353287    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 04 23:14:38.353883 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:38.353362    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 04 23:14:39.354152 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:39.354109    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 04 23:14:39.354705 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:39.354186    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 04 23:14:40.353327 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:40.353283    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 04 23:14:40.353536 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:40.353358    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 04 23:14:41.350441 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:41.350407    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:41.353518 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:41.353489    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 04 23:14:41.353759 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:41.353732    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 04 23:14:42.108708 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:42.108665    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd"]
Sep 04 23:14:42.112033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:42.112004    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd"
Sep 04 23:14:42.117666 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:42.116021    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:14:42.118920 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:42.118893    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd"]
Sep 04 23:14:42.286115 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:42.286081    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e834bd60-8a7d-4637-9d8c-06a142a2f2ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd\" (UID: \"e834bd60-8a7d-4637-9d8c-06a142a2f2ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd"
Sep 04 23:14:42.357656 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:42.357603    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:42.358101 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:42.358090    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:42.358934 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:42.358878    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:14:42.387366 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:42.387335    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e834bd60-8a7d-4637-9d8c-06a142a2f2ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd\" (UID: \"e834bd60-8a7d-4637-9d8c-06a142a2f2ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd"
Sep 04 23:14:42.387976 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:42.387953    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e834bd60-8a7d-4637-9d8c-06a142a2f2ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd\" (UID: \"e834bd60-8a7d-4637-9d8c-06a142a2f2ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd"
Sep 04 23:14:42.427773 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:42.427739    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd"
Sep 04 23:14:42.707213 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:42.707041    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd"]
Sep 04 23:14:42.710091 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:14:42.710067    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode834bd60_8a7d_4637_9d8c_06a142a2f2ab.slice/crio-4270fca270a037e28c260b92d38fa3843560b0c1eaea14632a42ba54103f9e8d WatchSource:0}: Error finding container 4270fca270a037e28c260b92d38fa3843560b0c1eaea14632a42ba54103f9e8d: Status 404 returned error can't find the container with id 4270fca270a037e28c260b92d38fa3843560b0c1eaea14632a42ba54103f9e8d
Sep 04 23:14:43.574152 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:43.574115    2578 generic.go:356] "Generic (PLEG): container finished" podID="e834bd60-8a7d-4637-9d8c-06a142a2f2ab" containerID="0a98586fd1686e18ece0a5c08686b0c992ccd260d482487795696dead4db9367" exitCode=0
Sep 04 23:14:43.574545 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:43.574203    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd" event={"ID":"e834bd60-8a7d-4637-9d8c-06a142a2f2ab","Type":"ContainerDied","Data":"0a98586fd1686e18ece0a5c08686b0c992ccd260d482487795696dead4db9367"}
Sep 04 23:14:43.574545 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:43.574257    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd" event={"ID":"e834bd60-8a7d-4637-9d8c-06a142a2f2ab","Type":"ContainerStarted","Data":"4270fca270a037e28c260b92d38fa3843560b0c1eaea14632a42ba54103f9e8d"}
Sep 04 23:14:44.713296 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:44.713269    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd"
Sep 04 23:14:44.720382 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:44.720359    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e834bd60-8a7d-4637-9d8c-06a142a2f2ab-maas-api-service-ca\") pod \"e834bd60-8a7d-4637-9d8c-06a142a2f2ab\" (UID: \"e834bd60-8a7d-4637-9d8c-06a142a2f2ab\") "
Sep 04 23:14:44.720685 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:44.720663    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e834bd60-8a7d-4637-9d8c-06a142a2f2ab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e834bd60-8a7d-4637-9d8c-06a142a2f2ab" (UID: "e834bd60-8a7d-4637-9d8c-06a142a2f2ab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:14:44.821197 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:44.821165    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e834bd60-8a7d-4637-9d8c-06a142a2f2ab-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:14:45.587274 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:45.587240    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd"
Sep 04 23:14:45.587451 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:45.587276    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd" event={"ID":"e834bd60-8a7d-4637-9d8c-06a142a2f2ab","Type":"ContainerDied","Data":"4270fca270a037e28c260b92d38fa3843560b0c1eaea14632a42ba54103f9e8d"}
Sep 04 23:14:45.587451 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:45.587325    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4270fca270a037e28c260b92d38fa3843560b0c1eaea14632a42ba54103f9e8d"
Sep 04 23:14:48.119568 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.119532    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:48.149408 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.149376    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"]
Sep 04 23:14:48.149694 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.149663    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt" podUID="7e4afce8-3b14-419b-8172-d85c2171b1a7" containerName="payload-processing" containerID="cri-o://a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf" gracePeriod=30
Sep 04 23:14:48.222572 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.222525    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:48.322365 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.322333    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:48.336434 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.336402    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd"]
Sep 04 23:14:48.339759 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.339735    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23158b-t42sd"]
Sep 04 23:14:48.381355 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.381282    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:48.430340 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.430256    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e834bd60-8a7d-4637-9d8c-06a142a2f2ab" path="/var/lib/kubelet/pods/e834bd60-8a7d-4637-9d8c-06a142a2f2ab/volumes"
Sep 04 23:14:48.432684 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.432660    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"
Sep 04 23:14:48.457912 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.457874    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8qjf\" (UniqueName: \"kubernetes.io/projected/7e4afce8-3b14-419b-8172-d85c2171b1a7-kube-api-access-b8qjf\") pod \"7e4afce8-3b14-419b-8172-d85c2171b1a7\" (UID: \"7e4afce8-3b14-419b-8172-d85c2171b1a7\") "
Sep 04 23:14:48.458056 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.457976    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4afce8-3b14-419b-8172-d85c2171b1a7-plugins-config-volume\") pod \"7e4afce8-3b14-419b-8172-d85c2171b1a7\" (UID: \"7e4afce8-3b14-419b-8172-d85c2171b1a7\") "
Sep 04 23:14:48.458325 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.458303    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e4afce8-3b14-419b-8172-d85c2171b1a7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7e4afce8-3b14-419b-8172-d85c2171b1a7" (UID: "7e4afce8-3b14-419b-8172-d85c2171b1a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:14:48.459954 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.459935    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e4afce8-3b14-419b-8172-d85c2171b1a7-kube-api-access-b8qjf" (OuterVolumeSpecName: "kube-api-access-b8qjf") pod "7e4afce8-3b14-419b-8172-d85c2171b1a7" (UID: "7e4afce8-3b14-419b-8172-d85c2171b1a7"). InnerVolumeSpecName "kube-api-access-b8qjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:14:48.559775 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.559744    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4afce8-3b14-419b-8172-d85c2171b1a7-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:14:48.559775 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.559773    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8qjf\" (UniqueName: \"kubernetes.io/projected/7e4afce8-3b14-419b-8172-d85c2171b1a7-kube-api-access-b8qjf\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:14:48.607422 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.607389    2578 generic.go:356] "Generic (PLEG): container finished" podID="7e4afce8-3b14-419b-8172-d85c2171b1a7" containerID="a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf" exitCode=0
Sep 04 23:14:48.607603 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.607445    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"
Sep 04 23:14:48.607603 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.607442    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt" event={"ID":"7e4afce8-3b14-419b-8172-d85c2171b1a7","Type":"ContainerDied","Data":"a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf"}
Sep 04 23:14:48.607603 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.607567    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt" event={"ID":"7e4afce8-3b14-419b-8172-d85c2171b1a7","Type":"ContainerDied","Data":"cc7b54a5be9c3376fd7376c525b38dea7a10c3d6f2abf5e47e530b8faf7026b7"}
Sep 04 23:14:48.607603 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.607587    2578 scope.go:117] "RemoveContainer" containerID="a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf"
Sep 04 23:14:48.624132 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.624100    2578 scope.go:117] "RemoveContainer" containerID="a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf"
Sep 04 23:14:48.624435 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:14:48.624411    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf\": container with ID starting with a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf not found: ID does not exist" containerID="a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf"
Sep 04 23:14:48.624505 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.624443    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf"} err="failed to get container status \"a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf\": rpc error: code = NotFound desc = could not find container \"a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf\": container with ID starting with a203abd30117f2f9a200b100d9b4cb64b3bff44264d3ce1162b4d894338d8ecf not found: ID does not exist"
Sep 04 23:14:48.633042 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.632979    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"]
Sep 04 23:14:48.638766 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.638742    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-23158b-5fb5bff869-2qwtt"]
Sep 04 23:14:48.689741 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:48.689717    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:50.420531 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:50.420496    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e4afce8-3b14-419b-8172-d85c2171b1a7" path="/var/lib/kubelet/pods/7e4afce8-3b14-419b-8172-d85c2171b1a7/volumes"
Sep 04 23:14:54.427941 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:54.427895    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:54.691891 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:54.691809    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"]
Sep 04 23:14:54.692054 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:54.692023    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy" containerID="cri-o://2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760" gracePeriod=30
Sep 04 23:14:55.813341 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:55.813307    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:57.360225 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:57.360185    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 04 23:14:57.360608 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:57.360250    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 04 23:14:58.507680 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.507617    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t"]
Sep 04 23:14:58.508082 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.508062    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e834bd60-8a7d-4637-9d8c-06a142a2f2ab" containerName="revoke-keys"
Sep 04 23:14:58.508082 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.508073    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e834bd60-8a7d-4637-9d8c-06a142a2f2ab" containerName="revoke-keys"
Sep 04 23:14:58.508182 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.508088    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4afce8-3b14-419b-8172-d85c2171b1a7" containerName="payload-processing"
Sep 04 23:14:58.508182 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.508094    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4afce8-3b14-419b-8172-d85c2171b1a7" containerName="payload-processing"
Sep 04 23:14:58.508182 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.508160    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e834bd60-8a7d-4637-9d8c-06a142a2f2ab" containerName="revoke-keys"
Sep 04 23:14:58.508182 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.508177    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4afce8-3b14-419b-8172-d85c2171b1a7" containerName="payload-processing"
Sep 04 23:14:58.512573 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.512554    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t"
Sep 04 23:14:58.516244 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.516218    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:14:58.519425 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.519398    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t"]
Sep 04 23:14:58.560321 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.560282    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t\" (UID: \"f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t"
Sep 04 23:14:58.628949 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.628915    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:14:58.661139 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.661103    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t\" (UID: \"f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t"
Sep 04 23:14:58.661754 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.661733    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t\" (UID: \"f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t"
Sep 04 23:14:58.824842 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:58.824809    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t"
Sep 04 23:14:59.102681 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:59.102634    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t"]
Sep 04 23:14:59.104074 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:14:59.104049    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b6d9a2_deef_4e0a_a0bd_ab189b5039d0.slice/crio-3e94247a0b37ab7fca32cc3f2f98d685efc389ab06f8822143bd7a16d98729b3 WatchSource:0}: Error finding container 3e94247a0b37ab7fca32cc3f2f98d685efc389ab06f8822143bd7a16d98729b3: Status 404 returned error can't find the container with id 3e94247a0b37ab7fca32cc3f2f98d685efc389ab06f8822143bd7a16d98729b3
Sep 04 23:14:59.690617 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:59.690518    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t" event={"ID":"f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0","Type":"ContainerStarted","Data":"20615c61222ddb9e9e9366158c59b5af68ebed3563bbc423e7fd0c710a844768"}
Sep 04 23:14:59.690617 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:59.690598    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t" event={"ID":"f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0","Type":"ContainerStarted","Data":"3e94247a0b37ab7fca32cc3f2f98d685efc389ab06f8822143bd7a16d98729b3"}
Sep 04 23:14:59.705409 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:14:59.705362    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t" podStartSLOduration=1.7053483950000001 podStartE2EDuration="1.705348395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:14:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:14:59.704375936 +0000 UTC m=+1787.814496526" watchObservedRunningTime="2026-09-04 23:14:59.705348395 +0000 UTC m=+1787.815468981"
Sep 04 23:15:00.002728 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.002700    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:15:00.075127 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075085    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-workload-certs\") pod \"72a952bf-e2fd-4eab-afe4-49b72725251c\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") "
Sep 04 23:15:00.075321 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075137    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-token\") pod \"72a952bf-e2fd-4eab-afe4-49b72725251c\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") "
Sep 04 23:15:00.075321 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075180    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-podinfo\") pod \"72a952bf-e2fd-4eab-afe4-49b72725251c\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") "
Sep 04 23:15:00.075321 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075202    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-data\") pod \"72a952bf-e2fd-4eab-afe4-49b72725251c\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") "
Sep 04 23:15:00.075321 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075230    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkk4j\" (UniqueName: \"kubernetes.io/projected/72a952bf-e2fd-4eab-afe4-49b72725251c-kube-api-access-vkk4j\") pod \"72a952bf-e2fd-4eab-afe4-49b72725251c\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") "
Sep 04 23:15:00.075321 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075266    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-envoy\") pod \"72a952bf-e2fd-4eab-afe4-49b72725251c\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") "
Sep 04 23:15:00.075321 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075294    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-workload-socket\") pod \"72a952bf-e2fd-4eab-afe4-49b72725251c\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") "
Sep 04 23:15:00.075636 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075374    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-credential-socket\") pod \"72a952bf-e2fd-4eab-afe4-49b72725251c\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") "
Sep 04 23:15:00.075636 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075415    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "72a952bf-e2fd-4eab-afe4-49b72725251c" (UID: "72a952bf-e2fd-4eab-afe4-49b72725251c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:15:00.075636 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075460    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72a952bf-e2fd-4eab-afe4-49b72725251c-istiod-ca-cert\") pod \"72a952bf-e2fd-4eab-afe4-49b72725251c\" (UID: \"72a952bf-e2fd-4eab-afe4-49b72725251c\") "
Sep 04 23:15:00.075636 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075537    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-data" (OuterVolumeSpecName: "istio-data") pod "72a952bf-e2fd-4eab-afe4-49b72725251c" (UID: "72a952bf-e2fd-4eab-afe4-49b72725251c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:15:00.075636 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075630    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "72a952bf-e2fd-4eab-afe4-49b72725251c" (UID: "72a952bf-e2fd-4eab-afe4-49b72725251c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:15:00.075958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075843    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:15:00.075958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075868    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:15:00.075958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075881    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:15:00.075958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075833    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "72a952bf-e2fd-4eab-afe4-49b72725251c" (UID: "72a952bf-e2fd-4eab-afe4-49b72725251c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:15:00.075958 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.075887    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72a952bf-e2fd-4eab-afe4-49b72725251c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "72a952bf-e2fd-4eab-afe4-49b72725251c" (UID: "72a952bf-e2fd-4eab-afe4-49b72725251c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:15:00.077592 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.077564    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "72a952bf-e2fd-4eab-afe4-49b72725251c" (UID: "72a952bf-e2fd-4eab-afe4-49b72725251c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:15:00.077698 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.077606    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72a952bf-e2fd-4eab-afe4-49b72725251c-kube-api-access-vkk4j" (OuterVolumeSpecName: "kube-api-access-vkk4j") pod "72a952bf-e2fd-4eab-afe4-49b72725251c" (UID: "72a952bf-e2fd-4eab-afe4-49b72725251c"). InnerVolumeSpecName "kube-api-access-vkk4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:15:00.077886 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.077866    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-token" (OuterVolumeSpecName: "istio-token") pod "72a952bf-e2fd-4eab-afe4-49b72725251c" (UID: "72a952bf-e2fd-4eab-afe4-49b72725251c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:15:00.077940 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.077891    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "72a952bf-e2fd-4eab-afe4-49b72725251c" (UID: "72a952bf-e2fd-4eab-afe4-49b72725251c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:15:00.135511 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.135463    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809395-4nj69"]
Sep 04 23:15:00.136143 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.136125    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy"
Sep 04 23:15:00.136195 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.136146    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy"
Sep 04 23:15:00.136195 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.136165    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy"
Sep 04 23:15:00.136195 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.136180    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy"
Sep 04 23:15:00.136297 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.136267    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy"
Sep 04 23:15:00.139965 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.139947    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-146c0e-29809395-j4h8w"]
Sep 04 23:15:00.140094 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.140074    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809395-4nj69"
Sep 04 23:15:00.140372 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.140355    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerName="istio-proxy"
Sep 04 23:15:00.143374 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.143354    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-146c0e-29809395-j4h8w"
Sep 04 23:15:00.150898 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.150122    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809395-4nj69"]
Sep 04 23:15:00.155925 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.155155    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-146c0e-29809395-j4h8w"]
Sep 04 23:15:00.160363 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.160341    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809395-4nj69"
Sep 04 23:15:00.166743 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.166724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-146c0e-29809395-j4h8w"
Sep 04 23:15:00.176581 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.176555    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:15:00.176703 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.176584    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72a952bf-e2fd-4eab-afe4-49b72725251c-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:15:00.176703 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.176599    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:15:00.176703 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.176611    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:15:00.176703 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.176625    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkk4j\" (UniqueName: \"kubernetes.io/projected/72a952bf-e2fd-4eab-afe4-49b72725251c-kube-api-access-vkk4j\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:15:00.176703 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.176664    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72a952bf-e2fd-4eab-afe4-49b72725251c-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:15:00.441980 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:15:00.441947    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda2463a_78ed_4bdb_a85f_d47ddda0099d.slice/crio-2eee66a44e1f5fe6549a88bea3d21e46acf5fb95966027c5f40471797a8b69e5 WatchSource:0}: Error finding container 2eee66a44e1f5fe6549a88bea3d21e46acf5fb95966027c5f40471797a8b69e5: Status 404 returned error can't find the container with id 2eee66a44e1f5fe6549a88bea3d21e46acf5fb95966027c5f40471797a8b69e5
Sep 04 23:15:00.442257 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.442215    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809395-4nj69"]
Sep 04 23:15:00.447365 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.447341    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-146c0e-29809395-j4h8w"]
Sep 04 23:15:00.447964 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:15:00.447944    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952c7536_f190_4b94_895b_845782683a18.slice/crio-3956239f2fb95806effc507bd46e4ae2340c2ce0fa0990c78e4994e19687c89c WatchSource:0}: Error finding container 3956239f2fb95806effc507bd46e4ae2340c2ce0fa0990c78e4994e19687c89c: Status 404 returned error can't find the container with id 3956239f2fb95806effc507bd46e4ae2340c2ce0fa0990c78e4994e19687c89c
Sep 04 23:15:00.698252 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.698213    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-146c0e-29809395-j4h8w" event={"ID":"952c7536-f190-4b94-895b-845782683a18","Type":"ContainerStarted","Data":"3956239f2fb95806effc507bd46e4ae2340c2ce0fa0990c78e4994e19687c89c"}
Sep 04 23:15:00.700060 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.700036    2578 generic.go:356] "Generic (PLEG): container finished" podID="72a952bf-e2fd-4eab-afe4-49b72725251c" containerID="2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760" exitCode=0
Sep 04 23:15:00.700198 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.700084    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"
Sep 04 23:15:00.700198 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.700119    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" event={"ID":"72a952bf-e2fd-4eab-afe4-49b72725251c","Type":"ContainerDied","Data":"2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760"}
Sep 04 23:15:00.700198 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.700165    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn" event={"ID":"72a952bf-e2fd-4eab-afe4-49b72725251c","Type":"ContainerDied","Data":"9761a5c99e8553b256d6879758ccfdf204668f2e441ce14eed255a5ab1eedc3b"}
Sep 04 23:15:00.700198 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.700198    2578 scope.go:117] "RemoveContainer" containerID="2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760"
Sep 04 23:15:00.702046 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.701949    2578 generic.go:356] "Generic (PLEG): container finished" podID="f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0" containerID="20615c61222ddb9e9e9366158c59b5af68ebed3563bbc423e7fd0c710a844768" exitCode=0
Sep 04 23:15:00.702046 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.701976    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t" event={"ID":"f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0","Type":"ContainerDied","Data":"20615c61222ddb9e9e9366158c59b5af68ebed3563bbc423e7fd0c710a844768"}
Sep 04 23:15:00.703696 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.703673    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809395-4nj69" event={"ID":"eda2463a-78ed-4bdb-a85f-d47ddda0099d","Type":"ContainerStarted","Data":"2eee66a44e1f5fe6549a88bea3d21e46acf5fb95966027c5f40471797a8b69e5"}
Sep 04 23:15:00.710513 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.710496    2578 scope.go:117] "RemoveContainer" containerID="cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c"
Sep 04 23:15:00.719760 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.719727    2578 scope.go:117] "RemoveContainer" containerID="2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760"
Sep 04 23:15:00.720008 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:15:00.719987    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760\": container with ID starting with 2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760 not found: ID does not exist" containerID="2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760"
Sep 04 23:15:00.720060 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.720017    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760"} err="failed to get container status \"2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760\": rpc error: code = NotFound desc = could not find container \"2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760\": container with ID starting with 2b4806356178630f1c13376bd2ae226a62405f69193b6613c035a03890379760 not found: ID does not exist"
Sep 04 23:15:00.720060 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.720034    2578 scope.go:117] "RemoveContainer" containerID="cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c"
Sep 04 23:15:00.720310 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:15:00.720292    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c\": container with ID starting with cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c not found: ID does not exist" containerID="cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c"
Sep 04 23:15:00.720371 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.720320    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c"} err="failed to get container status \"cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c\": rpc error: code = NotFound desc = could not find container \"cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c\": container with ID starting with cf971d08b9e3c9484b2aeabef2fbc9b991c4a365bea2fe3517cfa881512ab50c not found: ID does not exist"
Sep 04 23:15:00.738724 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.738687    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"]
Sep 04 23:15:00.741573 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:00.741549    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-23158b-openshift-default-64879469d6-lmkhn"]
Sep 04 23:15:01.712018 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:01.711931    2578 generic.go:356] "Generic (PLEG): container finished" podID="952c7536-f190-4b94-895b-845782683a18" containerID="375e655a60e1dc8d706e0a2907331971f5a4d777c7ca8d8c4ad24867535b1a52" exitCode=0
Sep 04 23:15:01.712419 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:01.712022    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-146c0e-29809395-j4h8w" event={"ID":"952c7536-f190-4b94-895b-845782683a18","Type":"ContainerDied","Data":"375e655a60e1dc8d706e0a2907331971f5a4d777c7ca8d8c4ad24867535b1a52"}
Sep 04 23:15:01.714650 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:01.714613    2578 generic.go:356] "Generic (PLEG): container finished" podID="eda2463a-78ed-4bdb-a85f-d47ddda0099d" containerID="0792b7f513b3e65df8497e0586043d371d2bcd89354d9ac6da0b0771e9d3cf6a" exitCode=0
Sep 04 23:15:01.714766 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:01.714667    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809395-4nj69" event={"ID":"eda2463a-78ed-4bdb-a85f-d47ddda0099d","Type":"ContainerDied","Data":"0792b7f513b3e65df8497e0586043d371d2bcd89354d9ac6da0b0771e9d3cf6a"}
Sep 04 23:15:01.855021 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:01.854992    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t"
Sep 04 23:15:01.996266 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:01.996168    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0-maas-api-service-ca\") pod \"f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0\" (UID: \"f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0\") "
Sep 04 23:15:01.996544 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:01.996519    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0" (UID: "f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:15:02.097387 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:02.097356    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:15:02.422674 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:02.421385    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72a952bf-e2fd-4eab-afe4-49b72725251c" path="/var/lib/kubelet/pods/72a952bf-e2fd-4eab-afe4-49b72725251c/volumes"
Sep 04 23:15:02.722700 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:02.722597    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t"
Sep 04 23:15:02.722700 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:02.722628    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t" event={"ID":"f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0","Type":"ContainerDied","Data":"3e94247a0b37ab7fca32cc3f2f98d685efc389ab06f8822143bd7a16d98729b3"}
Sep 04 23:15:02.723142 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:02.722711    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e94247a0b37ab7fca32cc3f2f98d685efc389ab06f8822143bd7a16d98729b3"
Sep 04 23:15:02.853043 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:02.852912    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:15:02.882030 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:02.882007    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-146c0e-29809395-j4h8w"
Sep 04 23:15:02.884088 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:02.884066    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809395-4nj69"
Sep 04 23:15:03.730374 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:03.730343    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-146c0e-29809395-j4h8w"
Sep 04 23:15:03.730787 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:03.730380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-146c0e-29809395-j4h8w" event={"ID":"952c7536-f190-4b94-895b-845782683a18","Type":"ContainerDied","Data":"3956239f2fb95806effc507bd46e4ae2340c2ce0fa0990c78e4994e19687c89c"}
Sep 04 23:15:03.730787 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:03.730426    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3956239f2fb95806effc507bd46e4ae2340c2ce0fa0990c78e4994e19687c89c"
Sep 04 23:15:03.731947 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:03.731930    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809395-4nj69"
Sep 04 23:15:03.732053 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:03.731971    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809395-4nj69" event={"ID":"eda2463a-78ed-4bdb-a85f-d47ddda0099d","Type":"ContainerDied","Data":"2eee66a44e1f5fe6549a88bea3d21e46acf5fb95966027c5f40471797a8b69e5"}
Sep 04 23:15:03.732053 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:03.732016    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2eee66a44e1f5fe6549a88bea3d21e46acf5fb95966027c5f40471797a8b69e5"
Sep 04 23:15:08.733283 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:08.733211    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:15:12.558085 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:12.558056    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 23:15:12.565999 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:12.565971    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 23:15:12.758595 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:12.758564    2578 scope.go:117] "RemoveContainer" containerID="e260abe9dacb187a89550bca5eb25637a8db2e1d63e86982b6d004c2b302be62"
Sep 04 23:15:14.298239 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.298204    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:15:14.332317 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.332284    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"]
Sep 04 23:15:14.332845 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.332808    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f" podUID="dc6d19fa-ad4c-4828-b804-3e6208ed17a1" containerName="payload-processing" containerID="cri-o://06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2" gracePeriod=30
Sep 04 23:15:14.399903 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.399877    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:15:14.498804 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.498766    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t"]
Sep 04 23:15:14.499557 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.499354    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:15:14.502515 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.502496    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3543cf-vk24t"]
Sep 04 23:15:14.559964 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.559868    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:15:14.591827 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.591805    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"
Sep 04 23:15:14.716794 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.716756    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2rkj\" (UniqueName: \"kubernetes.io/projected/dc6d19fa-ad4c-4828-b804-3e6208ed17a1-kube-api-access-c2rkj\") pod \"dc6d19fa-ad4c-4828-b804-3e6208ed17a1\" (UID: \"dc6d19fa-ad4c-4828-b804-3e6208ed17a1\") "
Sep 04 23:15:14.716998 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.716814    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc6d19fa-ad4c-4828-b804-3e6208ed17a1-plugins-config-volume\") pod \"dc6d19fa-ad4c-4828-b804-3e6208ed17a1\" (UID: \"dc6d19fa-ad4c-4828-b804-3e6208ed17a1\") "
Sep 04 23:15:14.717206 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.717177    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc6d19fa-ad4c-4828-b804-3e6208ed17a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc6d19fa-ad4c-4828-b804-3e6208ed17a1" (UID: "dc6d19fa-ad4c-4828-b804-3e6208ed17a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:15:14.719060 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.719035    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc6d19fa-ad4c-4828-b804-3e6208ed17a1-kube-api-access-c2rkj" (OuterVolumeSpecName: "kube-api-access-c2rkj") pod "dc6d19fa-ad4c-4828-b804-3e6208ed17a1" (UID: "dc6d19fa-ad4c-4828-b804-3e6208ed17a1"). InnerVolumeSpecName "kube-api-access-c2rkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:15:14.803922 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.803888    2578 generic.go:356] "Generic (PLEG): container finished" podID="dc6d19fa-ad4c-4828-b804-3e6208ed17a1" containerID="06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2" exitCode=0
Sep 04 23:15:14.804099 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.803939    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"
Sep 04 23:15:14.804099 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.803971    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f" event={"ID":"dc6d19fa-ad4c-4828-b804-3e6208ed17a1","Type":"ContainerDied","Data":"06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2"}
Sep 04 23:15:14.804099 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.804023    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f" event={"ID":"dc6d19fa-ad4c-4828-b804-3e6208ed17a1","Type":"ContainerDied","Data":"c9ae147b556f663227cc0afbb25e467d7d571ca6ed027864ca6efb329d3fc5e6"}
Sep 04 23:15:14.804099 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.804048    2578 scope.go:117] "RemoveContainer" containerID="06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2"
Sep 04 23:15:14.814066 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.814049    2578 scope.go:117] "RemoveContainer" containerID="06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2"
Sep 04 23:15:14.814320 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:15:14.814302    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2\": container with ID starting with 06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2 not found: ID does not exist" containerID="06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2"
Sep 04 23:15:14.814363 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.814329    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2"} err="failed to get container status \"06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2\": rpc error: code = NotFound desc = could not find container \"06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2\": container with ID starting with 06e845b0ca90a717ac368b12bc6a61b909f389f96275e87a625cefac678fb2c2 not found: ID does not exist"
Sep 04 23:15:14.818174 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.818154    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2rkj\" (UniqueName: \"kubernetes.io/projected/dc6d19fa-ad4c-4828-b804-3e6208ed17a1-kube-api-access-c2rkj\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:15:14.818242 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.818176    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc6d19fa-ad4c-4828-b804-3e6208ed17a1-plugins-config-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:15:14.826858 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.826828    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"]
Sep 04 23:15:14.830426 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.830405    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3543cf-769c5ff549-m882f"]
Sep 04 23:15:14.917099 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:14.917070    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:15:16.419484 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:16.419450    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc6d19fa-ad4c-4828-b804-3e6208ed17a1" path="/var/lib/kubelet/pods/dc6d19fa-ad4c-4828-b804-3e6208ed17a1/volumes"
Sep 04 23:15:16.419887 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:16.419839    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0" path="/var/lib/kubelet/pods/f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0/volumes"
Sep 04 23:15:19.989597 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:19.989549    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:15:32.180276 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:32.180243    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:15:47.592432 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:47.592390    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:15:47.649321 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:47.649290    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:15:56.262086 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:56.262028    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:15:56.710207 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:15:56.710178    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:02.084467 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.084417    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:02.184184 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.184140    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:02.375923 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.375836    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"]
Sep 04 23:16:02.376066 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.376043    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerName="istio-proxy" containerID="cri-o://aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512" gracePeriod=30
Sep 04 23:16:02.399312 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.399277    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:02.773791 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.773669    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l"]
Sep 04 23:16:02.774173 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.774154    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952c7536-f190-4b94-895b-845782683a18" containerName="cleanup"
Sep 04 23:16:02.774266 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.774176    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="952c7536-f190-4b94-895b-845782683a18" containerName="cleanup"
Sep 04 23:16:02.774266 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.774204    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0" containerName="revoke-keys"
Sep 04 23:16:02.774266 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.774213    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0" containerName="revoke-keys"
Sep 04 23:16:02.774266 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.774235    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc6d19fa-ad4c-4828-b804-3e6208ed17a1" containerName="payload-processing"
Sep 04 23:16:02.774266 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.774245    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc6d19fa-ad4c-4828-b804-3e6208ed17a1" containerName="payload-processing"
Sep 04 23:16:02.774266 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.774257    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eda2463a-78ed-4bdb-a85f-d47ddda0099d" containerName="cleanup"
Sep 04 23:16:02.774266 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.774265    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="eda2463a-78ed-4bdb-a85f-d47ddda0099d" containerName="cleanup"
Sep 04 23:16:02.774606 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.774355    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc6d19fa-ad4c-4828-b804-3e6208ed17a1" containerName="payload-processing"
Sep 04 23:16:02.774606 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.774375    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="eda2463a-78ed-4bdb-a85f-d47ddda0099d" containerName="cleanup"
Sep 04 23:16:02.774606 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.774387    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5b6d9a2-deef-4e0a-a0bd-ab189b5039d0" containerName="revoke-keys"
Sep 04 23:16:02.774606 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.774395    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="952c7536-f190-4b94-895b-845782683a18" containerName="cleanup"
Sep 04 23:16:02.777755 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.777734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l"
Sep 04 23:16:02.780226 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.780202    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:16:02.784306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.784283    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l"]
Sep 04 23:16:02.877594 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.877563    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93cef2de-156c-48c7-af3c-11ebeccb98bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l\" (UID: \"93cef2de-156c-48c7-af3c-11ebeccb98bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l"
Sep 04 23:16:02.978838 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.978801    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93cef2de-156c-48c7-af3c-11ebeccb98bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l\" (UID: \"93cef2de-156c-48c7-af3c-11ebeccb98bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l"
Sep 04 23:16:02.979427 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:02.979405    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93cef2de-156c-48c7-af3c-11ebeccb98bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l\" (UID: \"93cef2de-156c-48c7-af3c-11ebeccb98bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l"
Sep 04 23:16:03.027632 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:03.027552    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:03.090022 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:03.089971    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l"
Sep 04 23:16:03.370119 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:03.370090    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l"]
Sep 04 23:16:03.371357 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:16:03.371329    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93cef2de_156c_48c7_af3c_11ebeccb98bd.slice/crio-f6266c3a44cbebe2a87f3168a99fffd5040feed6c4e767cf2ea2319c8592b8a1 WatchSource:0}: Error finding container f6266c3a44cbebe2a87f3168a99fffd5040feed6c4e767cf2ea2319c8592b8a1: Status 404 returned error can't find the container with id f6266c3a44cbebe2a87f3168a99fffd5040feed6c4e767cf2ea2319c8592b8a1
Sep 04 23:16:04.122021 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:04.121985    2578 generic.go:356] "Generic (PLEG): container finished" podID="93cef2de-156c-48c7-af3c-11ebeccb98bd" containerID="a273e1d901b9189e49e0b7b98b2fba1b91ee31a86cadce3c06d65d13ac68e1be" exitCode=0
Sep 04 23:16:04.122442 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:04.122075    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l" event={"ID":"93cef2de-156c-48c7-af3c-11ebeccb98bd","Type":"ContainerDied","Data":"a273e1d901b9189e49e0b7b98b2fba1b91ee31a86cadce3c06d65d13ac68e1be"}
Sep 04 23:16:04.122442 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:04.122178    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l" event={"ID":"93cef2de-156c-48c7-af3c-11ebeccb98bd","Type":"ContainerStarted","Data":"f6266c3a44cbebe2a87f3168a99fffd5040feed6c4e767cf2ea2319c8592b8a1"}
Sep 04 23:16:05.263838 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:05.263808    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l"
Sep 04 23:16:05.301615 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:05.301585    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93cef2de-156c-48c7-af3c-11ebeccb98bd-maas-api-service-ca\") pod \"93cef2de-156c-48c7-af3c-11ebeccb98bd\" (UID: \"93cef2de-156c-48c7-af3c-11ebeccb98bd\") "
Sep 04 23:16:05.302033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:05.302003    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93cef2de-156c-48c7-af3c-11ebeccb98bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "93cef2de-156c-48c7-af3c-11ebeccb98bd" (UID: "93cef2de-156c-48c7-af3c-11ebeccb98bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:16:05.403049 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:05.402960    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93cef2de-156c-48c7-af3c-11ebeccb98bd-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:06.137279 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:06.137241    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l"
Sep 04 23:16:06.137468 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:06.137240    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l" event={"ID":"93cef2de-156c-48c7-af3c-11ebeccb98bd","Type":"ContainerDied","Data":"f6266c3a44cbebe2a87f3168a99fffd5040feed6c4e767cf2ea2319c8592b8a1"}
Sep 04 23:16:06.137468 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:06.137378    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6266c3a44cbebe2a87f3168a99fffd5040feed6c4e767cf2ea2319c8592b8a1"
Sep 04 23:16:07.639799 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.639737    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:16:07.723480 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.723438    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-token\") pod \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") "
Sep 04 23:16:07.723480 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.723490    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-data\") pod \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") "
Sep 04 23:16:07.723779 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.723510    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-workload-socket\") pod \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") "
Sep 04 23:16:07.723779 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.723536    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-credential-socket\") pod \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") "
Sep 04 23:16:07.723779 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.723569    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-podinfo\") pod \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") "
Sep 04 23:16:07.723779 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.723587    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-workload-certs\") pod \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") "
Sep 04 23:16:07.723779 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.723626    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istiod-ca-cert\") pod \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") "
Sep 04 23:16:07.723779 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.723714    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-envoy\") pod \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") "
Sep 04 23:16:07.723779 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.723761    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql8k7\" (UniqueName: \"kubernetes.io/projected/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-kube-api-access-ql8k7\") pod \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\" (UID: \"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf\") "
Sep 04 23:16:07.724140 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.723866    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" (UID: "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:16:07.724140 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.723948    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" (UID: "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:16:07.724140 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.724055    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" (UID: "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:16:07.724305 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.724206    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:07.724305 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.724225    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:07.724305 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.724241    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:07.724305 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.724253    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" (UID: "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:16:07.724458 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.724336    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" (UID: "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:16:07.726295 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.726268    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" (UID: "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:16:07.726295 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.726281    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" (UID: "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:16:07.726461 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.726394    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-kube-api-access-ql8k7" (OuterVolumeSpecName: "kube-api-access-ql8k7") pod "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" (UID: "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf"). InnerVolumeSpecName "kube-api-access-ql8k7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:16:07.726525 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.726504    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" (UID: "3b5e6d6e-7b61-4744-b9a7-ca7970553fcf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:16:07.824996 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.824967    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:07.824996 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.824993    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:07.824996 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.825002    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:07.825237 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.825010    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql8k7\" (UniqueName: \"kubernetes.io/projected/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-kube-api-access-ql8k7\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:07.825237 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.825019    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:07.825237 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:07.825028    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:08.151221 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.151185    2578 generic.go:356] "Generic (PLEG): container finished" podID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerID="aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512" exitCode=0
Sep 04 23:16:08.151424 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.151228    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" event={"ID":"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf","Type":"ContainerDied","Data":"aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512"}
Sep 04 23:16:08.151424 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.151237    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"
Sep 04 23:16:08.151424 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.151283    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs" event={"ID":"3b5e6d6e-7b61-4744-b9a7-ca7970553fcf","Type":"ContainerDied","Data":"d62a47d02041d6fa02203625817ed9fe19529c38835a2fd626a8a7dfc24254e2"}
Sep 04 23:16:08.151424 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.151311    2578 scope.go:117] "RemoveContainer" containerID="aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512"
Sep 04 23:16:08.161577 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.161554    2578 scope.go:117] "RemoveContainer" containerID="d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44"
Sep 04 23:16:08.170548 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.170527    2578 scope.go:117] "RemoveContainer" containerID="aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512"
Sep 04 23:16:08.170941 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:16:08.170903    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512\": container with ID starting with aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512 not found: ID does not exist" containerID="aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512"
Sep 04 23:16:08.171059 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.170952    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512"} err="failed to get container status \"aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512\": rpc error: code = NotFound desc = could not find container \"aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512\": container with ID starting with aa378f5136fe76db76574ba4124294cf43cef050f53ed05488b117648f8b0512 not found: ID does not exist"
Sep 04 23:16:08.171059 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.170976    2578 scope.go:117] "RemoveContainer" containerID="d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44"
Sep 04 23:16:08.171314 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:16:08.171290    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44\": container with ID starting with d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44 not found: ID does not exist" containerID="d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44"
Sep 04 23:16:08.171395 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.171318    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44"} err="failed to get container status \"d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44\": rpc error: code = NotFound desc = could not find container \"d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44\": container with ID starting with d4aafaf0a45d37e80f920d09af959bb2ecdc54d5b9618d100a02ec13624f1b44 not found: ID does not exist"
Sep 04 23:16:08.174127 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.174095    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"]
Sep 04 23:16:08.177043 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.177020    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-3543cf-openshift-default-6f7789d64b-hr7fs"]
Sep 04 23:16:08.422893 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:08.421154    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" path="/var/lib/kubelet/pods/3b5e6d6e-7b61-4744-b9a7-ca7970553fcf/volumes"
Sep 04 23:16:13.420114 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:13.420081    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:13.520186 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:13.520145    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:13.588437 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:13.588401    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:13.651439 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:13.651400    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l"]
Sep 04 23:16:13.652931 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:13.652897    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:13.654287 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:13.654265    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb64b4-bfs5l"]
Sep 04 23:16:13.717235 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:13.717157    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:14.037449 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:14.037364    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:14.425598 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:14.425554    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93cef2de-156c-48c7-af3c-11ebeccb98bd" path="/var/lib/kubelet/pods/93cef2de-156c-48c7-af3c-11ebeccb98bd/volumes"
Sep 04 23:16:19.705494 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:19.705460    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:19.807760 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:19.807723    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:19.980123 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:19.980034    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:20.311807 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.311713    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9"]
Sep 04 23:16:20.312153 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.312136    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerName="istio-proxy"
Sep 04 23:16:20.312205 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.312156    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerName="istio-proxy"
Sep 04 23:16:20.312205 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.312167    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerName="istio-proxy"
Sep 04 23:16:20.312205 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.312173    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerName="istio-proxy"
Sep 04 23:16:20.312205 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.312193    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93cef2de-156c-48c7-af3c-11ebeccb98bd" containerName="revoke-keys"
Sep 04 23:16:20.312205 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.312200    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="93cef2de-156c-48c7-af3c-11ebeccb98bd" containerName="revoke-keys"
Sep 04 23:16:20.312364 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.312256    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerName="istio-proxy"
Sep 04 23:16:20.312364 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.312267    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="93cef2de-156c-48c7-af3c-11ebeccb98bd" containerName="revoke-keys"
Sep 04 23:16:20.315492 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.315476    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9"
Sep 04 23:16:20.318268 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.318238    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:16:20.324570 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.324475    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9"]
Sep 04 23:16:20.414386 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.414345    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:20.442998 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.442962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a18a4ee-82de-4266-9517-ce2ac458a130-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9\" (UID: \"0a18a4ee-82de-4266-9517-ce2ac458a130\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9"
Sep 04 23:16:20.544563 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.544511    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a18a4ee-82de-4266-9517-ce2ac458a130-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9\" (UID: \"0a18a4ee-82de-4266-9517-ce2ac458a130\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9"
Sep 04 23:16:20.545171 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.545143    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a18a4ee-82de-4266-9517-ce2ac458a130-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9\" (UID: \"0a18a4ee-82de-4266-9517-ce2ac458a130\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9"
Sep 04 23:16:20.634077 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.634043    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9"
Sep 04 23:16:20.903973 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:20.903941    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9"]
Sep 04 23:16:20.905507 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:16:20.905480    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a18a4ee_82de_4266_9517_ce2ac458a130.slice/crio-3ed1dc0ef2519e5457827a1e616f7eaf00cf7a995d8f4e7752c76d97a2ac0a76 WatchSource:0}: Error finding container 3ed1dc0ef2519e5457827a1e616f7eaf00cf7a995d8f4e7752c76d97a2ac0a76: Status 404 returned error can't find the container with id 3ed1dc0ef2519e5457827a1e616f7eaf00cf7a995d8f4e7752c76d97a2ac0a76
Sep 04 23:16:21.231617 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:21.231524    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9" event={"ID":"0a18a4ee-82de-4266-9517-ce2ac458a130","Type":"ContainerStarted","Data":"3ed1dc0ef2519e5457827a1e616f7eaf00cf7a995d8f4e7752c76d97a2ac0a76"}
Sep 04 23:16:22.239261 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:22.239223    2578 generic.go:356] "Generic (PLEG): container finished" podID="0a18a4ee-82de-4266-9517-ce2ac458a130" containerID="61ddf4e44158df3ae296495d671e6561a6f00dc544d7cb9aa17e773e9e427a1c" exitCode=0
Sep 04 23:16:22.239724 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:22.239311    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9" event={"ID":"0a18a4ee-82de-4266-9517-ce2ac458a130","Type":"ContainerDied","Data":"61ddf4e44158df3ae296495d671e6561a6f00dc544d7cb9aa17e773e9e427a1c"}
Sep 04 23:16:23.375633 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:23.375609    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9"
Sep 04 23:16:23.471270 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:23.471238    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a18a4ee-82de-4266-9517-ce2ac458a130-maas-api-service-ca\") pod \"0a18a4ee-82de-4266-9517-ce2ac458a130\" (UID: \"0a18a4ee-82de-4266-9517-ce2ac458a130\") "
Sep 04 23:16:23.471593 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:23.471570    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a18a4ee-82de-4266-9517-ce2ac458a130-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a18a4ee-82de-4266-9517-ce2ac458a130" (UID: "0a18a4ee-82de-4266-9517-ce2ac458a130"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:16:23.572870 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:23.572780    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a18a4ee-82de-4266-9517-ce2ac458a130-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:24.251919 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:24.251894    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9"
Sep 04 23:16:24.252092 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:24.251944    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9" event={"ID":"0a18a4ee-82de-4266-9517-ce2ac458a130","Type":"ContainerDied","Data":"3ed1dc0ef2519e5457827a1e616f7eaf00cf7a995d8f4e7752c76d97a2ac0a76"}
Sep 04 23:16:24.252092 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:24.251993    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ed1dc0ef2519e5457827a1e616f7eaf00cf7a995d8f4e7752c76d97a2ac0a76"
Sep 04 23:16:30.804436 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:30.804399    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:30.903556 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:30.903524    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:30.967496 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:30.967460    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9"]
Sep 04 23:16:30.968733 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:30.968705    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:30.970049 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:30.970029    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ec1e5-t24k9"]
Sep 04 23:16:31.033575 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:31.033543    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:31.381628 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:31.381599    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:32.419524 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:32.419487    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a18a4ee-82de-4266-9517-ce2ac458a130" path="/var/lib/kubelet/pods/0a18a4ee-82de-4266-9517-ce2ac458a130/volumes"
Sep 04 23:16:37.096365 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.096290    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:37.165663 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.165608    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:37.343709 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.343674    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"]
Sep 04 23:16:37.343902 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.343867    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy" containerID="cri-o://c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1" gracePeriod=30
Sep 04 23:16:37.365666 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.365580    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:37.713339 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.713234    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr"]
Sep 04 23:16:37.714439 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.714406    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a18a4ee-82de-4266-9517-ce2ac458a130" containerName="revoke-keys"
Sep 04 23:16:37.714603 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.714593    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a18a4ee-82de-4266-9517-ce2ac458a130" containerName="revoke-keys"
Sep 04 23:16:37.714930 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.714914    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a18a4ee-82de-4266-9517-ce2ac458a130" containerName="revoke-keys"
Sep 04 23:16:37.715050 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.715038    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b5e6d6e-7b61-4744-b9a7-ca7970553fcf" containerName="istio-proxy"
Sep 04 23:16:37.719191 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.719159    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr"
Sep 04 23:16:37.721223 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.721202    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr"]
Sep 04 23:16:37.721853 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.721833    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ncjwg\""
Sep 04 23:16:37.809429 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.809385    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ffcd42e-7118-4e52-a921-12c72691e483-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr\" (UID: \"0ffcd42e-7118-4e52-a921-12c72691e483\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr"
Sep 04 23:16:37.901499 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.901466    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:37.910676 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.910628    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ffcd42e-7118-4e52-a921-12c72691e483-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr\" (UID: \"0ffcd42e-7118-4e52-a921-12c72691e483\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr"
Sep 04 23:16:37.911216 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:37.911195    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ffcd42e-7118-4e52-a921-12c72691e483-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr\" (UID: \"0ffcd42e-7118-4e52-a921-12c72691e483\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr"
Sep 04 23:16:38.031526 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:38.031429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr"
Sep 04 23:16:38.304426 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:38.304393    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr"]
Sep 04 23:16:38.305447 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:16:38.305423    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ffcd42e_7118_4e52_a921_12c72691e483.slice/crio-866af59626ad72f8f4aa6299dc001216dcef0badf5f675cf48b1923bcd4f1775 WatchSource:0}: Error finding container 866af59626ad72f8f4aa6299dc001216dcef0badf5f675cf48b1923bcd4f1775: Status 404 returned error can't find the container with id 866af59626ad72f8f4aa6299dc001216dcef0badf5f675cf48b1923bcd4f1775
Sep 04 23:16:38.341585 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:38.341552    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr" event={"ID":"0ffcd42e-7118-4e52-a921-12c72691e483","Type":"ContainerStarted","Data":"866af59626ad72f8f4aa6299dc001216dcef0badf5f675cf48b1923bcd4f1775"}
Sep 04 23:16:39.350411 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:39.350374    2578 generic.go:356] "Generic (PLEG): container finished" podID="0ffcd42e-7118-4e52-a921-12c72691e483" containerID="0f48c9fda25d1fef0da825bc58841743d4fb9489f2e6ee171a66b0d96372d09c" exitCode=0
Sep 04 23:16:39.350818 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:39.350458    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr" event={"ID":"0ffcd42e-7118-4e52-a921-12c72691e483","Type":"ContainerDied","Data":"0f48c9fda25d1fef0da825bc58841743d4fb9489f2e6ee171a66b0d96372d09c"}
Sep 04 23:16:40.489922 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:40.489897    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr"
Sep 04 23:16:40.637203 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:40.637100    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ffcd42e-7118-4e52-a921-12c72691e483-maas-api-service-ca\") pod \"0ffcd42e-7118-4e52-a921-12c72691e483\" (UID: \"0ffcd42e-7118-4e52-a921-12c72691e483\") "
Sep 04 23:16:40.637559 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:40.637529    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ffcd42e-7118-4e52-a921-12c72691e483-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ffcd42e-7118-4e52-a921-12c72691e483" (UID: "0ffcd42e-7118-4e52-a921-12c72691e483"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:16:40.738519 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:40.738477    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ffcd42e-7118-4e52-a921-12c72691e483-maas-api-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:41.367755 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:41.367726    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr"
Sep 04 23:16:41.367935 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:41.367770    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr" event={"ID":"0ffcd42e-7118-4e52-a921-12c72691e483","Type":"ContainerDied","Data":"866af59626ad72f8f4aa6299dc001216dcef0badf5f675cf48b1923bcd4f1775"}
Sep 04 23:16:41.367935 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:41.367824    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="866af59626ad72f8f4aa6299dc001216dcef0badf5f675cf48b1923bcd4f1775"
Sep 04 23:16:42.584768 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.584739    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:16:42.756989 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.756885    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a6102b2-41e7-4846-a636-015af1be4912-istiod-ca-cert\") pod \"9a6102b2-41e7-4846-a636-015af1be4912\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") "
Sep 04 23:16:42.756989 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.756949    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-credential-socket\") pod \"9a6102b2-41e7-4846-a636-015af1be4912\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") "
Sep 04 23:16:42.757218 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.756996    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a6102b2-41e7-4846-a636-015af1be4912-istio-podinfo\") pod \"9a6102b2-41e7-4846-a636-015af1be4912\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") "
Sep 04 23:16:42.757218 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757040    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-workload-certs\") pod \"9a6102b2-41e7-4846-a636-015af1be4912\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") "
Sep 04 23:16:42.757218 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757070    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-istio-envoy\") pod \"9a6102b2-41e7-4846-a636-015af1be4912\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") "
Sep 04 23:16:42.757218 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757122    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-workload-socket\") pod \"9a6102b2-41e7-4846-a636-015af1be4912\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") "
Sep 04 23:16:42.757218 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757145    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-istio-data\") pod \"9a6102b2-41e7-4846-a636-015af1be4912\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") "
Sep 04 23:16:42.757218 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757196    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cdww\" (UniqueName: \"kubernetes.io/projected/9a6102b2-41e7-4846-a636-015af1be4912-kube-api-access-2cdww\") pod \"9a6102b2-41e7-4846-a636-015af1be4912\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") "
Sep 04 23:16:42.757515 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757227    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a6102b2-41e7-4846-a636-015af1be4912-istio-token\") pod \"9a6102b2-41e7-4846-a636-015af1be4912\" (UID: \"9a6102b2-41e7-4846-a636-015af1be4912\") "
Sep 04 23:16:42.757515 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757312    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9a6102b2-41e7-4846-a636-015af1be4912" (UID: "9a6102b2-41e7-4846-a636-015af1be4912"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:16:42.757515 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757346    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9a6102b2-41e7-4846-a636-015af1be4912" (UID: "9a6102b2-41e7-4846-a636-015af1be4912"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:16:42.757515 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757424    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-istio-data" (OuterVolumeSpecName: "istio-data") pod "9a6102b2-41e7-4846-a636-015af1be4912" (UID: "9a6102b2-41e7-4846-a636-015af1be4912"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:16:42.757515 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757347    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a6102b2-41e7-4846-a636-015af1be4912-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9a6102b2-41e7-4846-a636-015af1be4912" (UID: "9a6102b2-41e7-4846-a636-015af1be4912"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:16:42.757515 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757443    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9a6102b2-41e7-4846-a636-015af1be4912" (UID: "9a6102b2-41e7-4846-a636-015af1be4912"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:16:42.757816 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757616    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:42.757816 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757634    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:42.757816 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757677    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:42.757816 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757690    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a6102b2-41e7-4846-a636-015af1be4912-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:42.757816 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.757702    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:42.759371 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.759338    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9a6102b2-41e7-4846-a636-015af1be4912" (UID: "9a6102b2-41e7-4846-a636-015af1be4912"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:16:42.759500 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.759376    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9a6102b2-41e7-4846-a636-015af1be4912-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9a6102b2-41e7-4846-a636-015af1be4912" (UID: "9a6102b2-41e7-4846-a636-015af1be4912"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:16:42.759792 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.759765    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a6102b2-41e7-4846-a636-015af1be4912-kube-api-access-2cdww" (OuterVolumeSpecName: "kube-api-access-2cdww") pod "9a6102b2-41e7-4846-a636-015af1be4912" (UID: "9a6102b2-41e7-4846-a636-015af1be4912"). InnerVolumeSpecName "kube-api-access-2cdww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:16:42.759792 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.759775    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a6102b2-41e7-4846-a636-015af1be4912-istio-token" (OuterVolumeSpecName: "istio-token") pod "9a6102b2-41e7-4846-a636-015af1be4912" (UID: "9a6102b2-41e7-4846-a636-015af1be4912"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:16:42.858683 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.858631    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cdww\" (UniqueName: \"kubernetes.io/projected/9a6102b2-41e7-4846-a636-015af1be4912-kube-api-access-2cdww\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:42.858683 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.858683    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a6102b2-41e7-4846-a636-015af1be4912-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:42.858875 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.858697    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a6102b2-41e7-4846-a636-015af1be4912-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:42.858875 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:42.858705    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a6102b2-41e7-4846-a636-015af1be4912-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:16:43.381013 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:43.380980    2578 generic.go:356] "Generic (PLEG): container finished" podID="9a6102b2-41e7-4846-a636-015af1be4912" containerID="c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1" exitCode=0
Sep 04 23:16:43.381217 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:43.381026    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"
Sep 04 23:16:43.381217 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:43.381068    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" event={"ID":"9a6102b2-41e7-4846-a636-015af1be4912","Type":"ContainerDied","Data":"c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1"}
Sep 04 23:16:43.381217 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:43.381127    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj" event={"ID":"9a6102b2-41e7-4846-a636-015af1be4912","Type":"ContainerDied","Data":"2805606dd3e79e7a02fb2f23e9559146913c1f623ab90e567636bab9fdb30668"}
Sep 04 23:16:43.381217 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:43.381156    2578 scope.go:117] "RemoveContainer" containerID="c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1"
Sep 04 23:16:43.391036 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:43.391018    2578 scope.go:117] "RemoveContainer" containerID="ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b"
Sep 04 23:16:43.399781 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:43.399759    2578 scope.go:117] "RemoveContainer" containerID="c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1"
Sep 04 23:16:43.400146 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:16:43.400120    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1\": container with ID starting with c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1 not found: ID does not exist" containerID="c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1"
Sep 04 23:16:43.400282 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:43.400156    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1"} err="failed to get container status \"c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1\": rpc error: code = NotFound desc = could not find container \"c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1\": container with ID starting with c78943db53baa9ec91a6b8e963479d5adb5b69c7714fb8acf4f7ba10aee01ee1 not found: ID does not exist"
Sep 04 23:16:43.400282 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:43.400188    2578 scope.go:117] "RemoveContainer" containerID="ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b"
Sep 04 23:16:43.400526 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:16:43.400508    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b\": container with ID starting with ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b not found: ID does not exist" containerID="ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b"
Sep 04 23:16:43.400573 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:43.400533    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b"} err="failed to get container status \"ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b\": rpc error: code = NotFound desc = could not find container \"ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b\": container with ID starting with ac55fb6622fa9519010e92d230f03d3a51f86eaae3cf674d20e153b2af88048b not found: ID does not exist"
Sep 04 23:16:43.402770 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:43.402753    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"]
Sep 04 23:16:43.406402 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:43.406380    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-9ec1e5-openshift-default-79964db456-cdgkj"]
Sep 04 23:16:44.419975 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:44.419941    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a6102b2-41e7-4846-a636-015af1be4912" path="/var/lib/kubelet/pods/9a6102b2-41e7-4846-a636-015af1be4912/volumes"
Sep 04 23:16:48.236120 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:48.236079    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-146c0e-29809395-j4h8w"]
Sep 04 23:16:48.236601 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:48.236577    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:48.240693 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:48.240667    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-146c0e-29809395-j4h8w"]
Sep 04 23:16:48.334257 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:48.334225    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:48.419986 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:48.419950    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="952c7536-f190-4b94-895b-845782683a18" path="/var/lib/kubelet/pods/952c7536-f190-4b94-895b-845782683a18/volumes"
Sep 04 23:16:48.426069 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:48.426044    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr"]
Sep 04 23:16:48.429445 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:48.429423    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-146c0e-2qccr"]
Sep 04 23:16:48.429846 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:48.429828    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:48.494613 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:48.494526    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:48.811976 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:48.811892    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:50.420495 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:50.420462    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ffcd42e-7118-4e52-a921-12c72691e483" path="/var/lib/kubelet/pods/0ffcd42e-7118-4e52-a921-12c72691e483/volumes"
Sep 04 23:16:54.542594 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:54.542564    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:54.642816 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:54.642777    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:16:54.799269 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:54.799192    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"]
Sep 04 23:16:54.799484 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:54.799458    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerName="istio-proxy" containerID="cri-o://a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb" gracePeriod=30
Sep 04 23:16:54.827244 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:16:54.827210    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:17:00.049219 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.049193    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:17:00.099684 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.099633    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5425fba2-c853-480c-b386-1ae2b4dbfc40-istiod-ca-cert\") pod \"5425fba2-c853-480c-b386-1ae2b4dbfc40\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") "
Sep 04 23:17:00.099848 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.099773    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fggwb\" (UniqueName: \"kubernetes.io/projected/5425fba2-c853-480c-b386-1ae2b4dbfc40-kube-api-access-fggwb\") pod \"5425fba2-c853-480c-b386-1ae2b4dbfc40\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") "
Sep 04 23:17:00.099848 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.099806    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-data\") pod \"5425fba2-c853-480c-b386-1ae2b4dbfc40\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") "
Sep 04 23:17:00.099848 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.099834    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-workload-certs\") pod \"5425fba2-c853-480c-b386-1ae2b4dbfc40\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") "
Sep 04 23:17:00.100038 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.099869    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-workload-socket\") pod \"5425fba2-c853-480c-b386-1ae2b4dbfc40\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") "
Sep 04 23:17:00.100038 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.099919    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-token\") pod \"5425fba2-c853-480c-b386-1ae2b4dbfc40\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") "
Sep 04 23:17:00.100038 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.099965    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-credential-socket\") pod \"5425fba2-c853-480c-b386-1ae2b4dbfc40\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") "
Sep 04 23:17:00.100038 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.099996    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-envoy\") pod \"5425fba2-c853-480c-b386-1ae2b4dbfc40\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") "
Sep 04 23:17:00.100038 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.100026    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-podinfo\") pod \"5425fba2-c853-480c-b386-1ae2b4dbfc40\" (UID: \"5425fba2-c853-480c-b386-1ae2b4dbfc40\") "
Sep 04 23:17:00.100303 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.100035    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5425fba2-c853-480c-b386-1ae2b4dbfc40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5425fba2-c853-480c-b386-1ae2b4dbfc40" (UID: "5425fba2-c853-480c-b386-1ae2b4dbfc40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:17:00.100303 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.100126    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-data" (OuterVolumeSpecName: "istio-data") pod "5425fba2-c853-480c-b386-1ae2b4dbfc40" (UID: "5425fba2-c853-480c-b386-1ae2b4dbfc40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:17:00.100303 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.100150    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5425fba2-c853-480c-b386-1ae2b4dbfc40" (UID: "5425fba2-c853-480c-b386-1ae2b4dbfc40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:17:00.100303 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.100241    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5425fba2-c853-480c-b386-1ae2b4dbfc40" (UID: "5425fba2-c853-480c-b386-1ae2b4dbfc40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:17:00.100514 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.100356    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5425fba2-c853-480c-b386-1ae2b4dbfc40" (UID: "5425fba2-c853-480c-b386-1ae2b4dbfc40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:17:00.100514 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.100428    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-credential-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:17:00.100514 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.100443    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5425fba2-c853-480c-b386-1ae2b4dbfc40-istiod-ca-cert\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:17:00.100514 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.100453    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-data\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:17:00.100514 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.100464    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-workload-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:17:00.100514 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.100472    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-workload-socket\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:17:00.102327 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.102278    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5425fba2-c853-480c-b386-1ae2b4dbfc40" (UID: "5425fba2-c853-480c-b386-1ae2b4dbfc40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:17:00.102460 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.102370    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5425fba2-c853-480c-b386-1ae2b4dbfc40" (UID: "5425fba2-c853-480c-b386-1ae2b4dbfc40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:17:00.102524 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.102470    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5425fba2-c853-480c-b386-1ae2b4dbfc40-kube-api-access-fggwb" (OuterVolumeSpecName: "kube-api-access-fggwb") pod "5425fba2-c853-480c-b386-1ae2b4dbfc40" (UID: "5425fba2-c853-480c-b386-1ae2b4dbfc40"). InnerVolumeSpecName "kube-api-access-fggwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:17:00.102524 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.102488    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-token" (OuterVolumeSpecName: "istio-token") pod "5425fba2-c853-480c-b386-1ae2b4dbfc40" (UID: "5425fba2-c853-480c-b386-1ae2b4dbfc40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:17:00.200894 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.200860    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fggwb\" (UniqueName: \"kubernetes.io/projected/5425fba2-c853-480c-b386-1ae2b4dbfc40-kube-api-access-fggwb\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:17:00.200894 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.200889    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-token\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:17:00.200894 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.200898    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-envoy\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:17:00.200894 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.200905    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5425fba2-c853-480c-b386-1ae2b4dbfc40-istio-podinfo\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:17:00.489084 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.489052    2578 generic.go:356] "Generic (PLEG): container finished" podID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerID="a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb" exitCode=0
Sep 04 23:17:00.489233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.489093    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"
Sep 04 23:17:00.489233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.489113    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" event={"ID":"5425fba2-c853-480c-b386-1ae2b4dbfc40","Type":"ContainerDied","Data":"a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb"}
Sep 04 23:17:00.489233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.489161    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns" event={"ID":"5425fba2-c853-480c-b386-1ae2b4dbfc40","Type":"ContainerDied","Data":"17924ffa1c7955e8d7320dfb48ac80f189fedcd756ac8afbcfed99f98ea248ce"}
Sep 04 23:17:00.489233 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.489186    2578 scope.go:117] "RemoveContainer" containerID="a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb"
Sep 04 23:17:00.498526 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.498503    2578 scope.go:117] "RemoveContainer" containerID="f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340"
Sep 04 23:17:00.507588 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.507565    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"]
Sep 04 23:17:00.508321 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.508292    2578 scope.go:117] "RemoveContainer" containerID="a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb"
Sep 04 23:17:00.508628 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:17:00.508608    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb\": container with ID starting with a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb not found: ID does not exist" containerID="a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb"
Sep 04 23:17:00.508974 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.508637    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb"} err="failed to get container status \"a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb\": rpc error: code = NotFound desc = could not find container \"a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb\": container with ID starting with a70e6f4a7ed4d13fa5343dc57fe9611af8c8dad3ebb8efb1b7dbb68f2da65bfb not found: ID does not exist"
Sep 04 23:17:00.508974 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.508685    2578 scope.go:117] "RemoveContainer" containerID="f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340"
Sep 04 23:17:00.509099 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:17:00.509013    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340\": container with ID starting with f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340 not found: ID does not exist" containerID="f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340"
Sep 04 23:17:00.509099 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.509044    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340"} err="failed to get container status \"f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340\": rpc error: code = NotFound desc = could not find container \"f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340\": container with ID starting with f91f2ed4cf0e3ca332caa854f5ef1fa3855c1f729498382c671dcdaeb64c1340 not found: ID does not exist"
Sep 04 23:17:00.511851 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:00.511826    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-146c0e-openshift-default-cf9d4646d-cdsns"]
Sep 04 23:17:02.419306 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:02.419271    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" path="/var/lib/kubelet/pods/5425fba2-c853-480c-b386-1ae2b4dbfc40/volumes"
Sep 04 23:17:12.809038 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:17:12.809005    2578 scope.go:117] "RemoveContainer" containerID="2d68b50ff4412f4d05357088eb9d0ed72c98d8e6cd8b2ad05d79a13883579cea"
Sep 04 23:18:12.840184 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:18:12.840146    2578 scope.go:117] "RemoveContainer" containerID="cd6be43a6cce8e7b1e4a9be28782d6d9eca48c36b59fedc6111793ca439ba2a4"
Sep 04 23:18:19.232769 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:18:19.232732    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:18:20.919617 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:18:20.919583    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:18:22.422576 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:18:22.422543    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:19:12.863029 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:12.862997    2578 scope.go:117] "RemoveContainer" containerID="0f687e177d18deaa76b51d9ffd24f1a86975a9a5d78ee436100b9a837a0b61a0"
Sep 04 23:19:34.326908 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:34.326875    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:19:34.572321 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:34.572287    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"]
Sep 04 23:19:34.572507 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:34.572458    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h" podUID="65624aef-fcac-43d9-86e6-2c6a3d41c4ee" containerName="manager" containerID="cri-o://4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7" gracePeriod=10
Sep 04 23:19:36.933033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:36.933005    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"
Sep 04 23:19:37.005910 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.005825    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgg2m\" (UniqueName: \"kubernetes.io/projected/65624aef-fcac-43d9-86e6-2c6a3d41c4ee-kube-api-access-hgg2m\") pod \"65624aef-fcac-43d9-86e6-2c6a3d41c4ee\" (UID: \"65624aef-fcac-43d9-86e6-2c6a3d41c4ee\") "
Sep 04 23:19:37.005910 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.005881    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/65624aef-fcac-43d9-86e6-2c6a3d41c4ee-extensions-socket-volume\") pod \"65624aef-fcac-43d9-86e6-2c6a3d41c4ee\" (UID: \"65624aef-fcac-43d9-86e6-2c6a3d41c4ee\") "
Sep 04 23:19:37.006321 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.006292    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65624aef-fcac-43d9-86e6-2c6a3d41c4ee-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "65624aef-fcac-43d9-86e6-2c6a3d41c4ee" (UID: "65624aef-fcac-43d9-86e6-2c6a3d41c4ee"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:19:37.007892 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.007864    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65624aef-fcac-43d9-86e6-2c6a3d41c4ee-kube-api-access-hgg2m" (OuterVolumeSpecName: "kube-api-access-hgg2m") pod "65624aef-fcac-43d9-86e6-2c6a3d41c4ee" (UID: "65624aef-fcac-43d9-86e6-2c6a3d41c4ee"). InnerVolumeSpecName "kube-api-access-hgg2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:19:37.106535 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.106499    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgg2m\" (UniqueName: \"kubernetes.io/projected/65624aef-fcac-43d9-86e6-2c6a3d41c4ee-kube-api-access-hgg2m\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:19:37.106535 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.106532    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/65624aef-fcac-43d9-86e6-2c6a3d41c4ee-extensions-socket-volume\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:19:37.426359 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.426323    2578 generic.go:356] "Generic (PLEG): container finished" podID="65624aef-fcac-43d9-86e6-2c6a3d41c4ee" containerID="4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7" exitCode=0
Sep 04 23:19:37.426535 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.426365    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"
Sep 04 23:19:37.426535 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.426403    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h" event={"ID":"65624aef-fcac-43d9-86e6-2c6a3d41c4ee","Type":"ContainerDied","Data":"4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7"}
Sep 04 23:19:37.426535 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.426459    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h" event={"ID":"65624aef-fcac-43d9-86e6-2c6a3d41c4ee","Type":"ContainerDied","Data":"9b5e2ee6ff40bca698ad0b0cd1c4ee82e34eac2fb92a18d998b1aa17c4fdb1f7"}
Sep 04 23:19:37.426535 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.426481    2578 scope.go:117] "RemoveContainer" containerID="4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7"
Sep 04 23:19:37.436931 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.436914    2578 scope.go:117] "RemoveContainer" containerID="4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7"
Sep 04 23:19:37.437193 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:19:37.437171    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7\": container with ID starting with 4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7 not found: ID does not exist" containerID="4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7"
Sep 04 23:19:37.437240 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.437202    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7"} err="failed to get container status \"4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7\": rpc error: code = NotFound desc = could not find container \"4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7\": container with ID starting with 4f7ca633c95b9f83638caa93f4efe4a6e8fdabda886122261c3ed01738b3a7e7 not found: ID does not exist"
Sep 04 23:19:37.448627 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.448595    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"]
Sep 04 23:19:37.452050 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:37.452027    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-qtz9h"]
Sep 04 23:19:38.420495 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:19:38.420463    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65624aef-fcac-43d9-86e6-2c6a3d41c4ee" path="/var/lib/kubelet/pods/65624aef-fcac-43d9-86e6-2c6a3d41c4ee/volumes"
Sep 04 23:20:12.598545 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:12.598513    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 23:20:12.607702 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:12.607677    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 23:20:35.310076 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310039    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"]
Sep 04 23:20:35.310464 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310443    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerName="istio-proxy"
Sep 04 23:20:35.310464 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310455    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerName="istio-proxy"
Sep 04 23:20:35.310539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310470    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy"
Sep 04 23:20:35.310539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310476    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy"
Sep 04 23:20:35.310539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310494    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65624aef-fcac-43d9-86e6-2c6a3d41c4ee" containerName="manager"
Sep 04 23:20:35.310539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310502    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="65624aef-fcac-43d9-86e6-2c6a3d41c4ee" containerName="manager"
Sep 04 23:20:35.310539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310510    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy"
Sep 04 23:20:35.310539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310515    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy"
Sep 04 23:20:35.310539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310524    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ffcd42e-7118-4e52-a921-12c72691e483" containerName="revoke-keys"
Sep 04 23:20:35.310539 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310530    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ffcd42e-7118-4e52-a921-12c72691e483" containerName="revoke-keys"
Sep 04 23:20:35.310829 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310595    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerName="istio-proxy"
Sep 04 23:20:35.310829 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310605    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ffcd42e-7118-4e52-a921-12c72691e483" containerName="revoke-keys"
Sep 04 23:20:35.310829 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310611    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy"
Sep 04 23:20:35.310829 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310617    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="65624aef-fcac-43d9-86e6-2c6a3d41c4ee" containerName="manager"
Sep 04 23:20:35.310829 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310623    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerName="istio-proxy"
Sep 04 23:20:35.310829 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.310628    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a6102b2-41e7-4846-a636-015af1be4912" containerName="istio-proxy"
Sep 04 23:20:35.313648 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.313621    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"
Sep 04 23:20:35.316423 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.316398    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rs8mg\""
Sep 04 23:20:35.326731 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.326696    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"]
Sep 04 23:20:35.429884 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.429853    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngjtc\" (UniqueName: \"kubernetes.io/projected/259bbe51-fb89-430d-b80f-f77c2af51cc7-kube-api-access-ngjtc\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-ktxw6\" (UID: \"259bbe51-fb89-430d-b80f-f77c2af51cc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"
Sep 04 23:20:35.430081 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.429919    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/259bbe51-fb89-430d-b80f-f77c2af51cc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-ktxw6\" (UID: \"259bbe51-fb89-430d-b80f-f77c2af51cc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"
Sep 04 23:20:35.530877 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.530832    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/259bbe51-fb89-430d-b80f-f77c2af51cc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-ktxw6\" (UID: \"259bbe51-fb89-430d-b80f-f77c2af51cc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"
Sep 04 23:20:35.531059 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.530922    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngjtc\" (UniqueName: \"kubernetes.io/projected/259bbe51-fb89-430d-b80f-f77c2af51cc7-kube-api-access-ngjtc\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-ktxw6\" (UID: \"259bbe51-fb89-430d-b80f-f77c2af51cc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"
Sep 04 23:20:35.531215 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.531192    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/259bbe51-fb89-430d-b80f-f77c2af51cc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-ktxw6\" (UID: \"259bbe51-fb89-430d-b80f-f77c2af51cc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"
Sep 04 23:20:35.539164 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.539136    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngjtc\" (UniqueName: \"kubernetes.io/projected/259bbe51-fb89-430d-b80f-f77c2af51cc7-kube-api-access-ngjtc\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-ktxw6\" (UID: \"259bbe51-fb89-430d-b80f-f77c2af51cc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"
Sep 04 23:20:35.624454 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.624419    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"
Sep 04 23:20:35.905085 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:35.905050    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"]
Sep 04 23:20:35.906757 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:20:35.906731    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod259bbe51_fb89_430d_b80f_f77c2af51cc7.slice/crio-5902b924989835b178080bc0c1f54c8de2e1a38bc3ade3ba8de484e9f026a535 WatchSource:0}: Error finding container 5902b924989835b178080bc0c1f54c8de2e1a38bc3ade3ba8de484e9f026a535: Status 404 returned error can't find the container with id 5902b924989835b178080bc0c1f54c8de2e1a38bc3ade3ba8de484e9f026a535
Sep 04 23:20:36.762714 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:36.762677    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6" event={"ID":"259bbe51-fb89-430d-b80f-f77c2af51cc7","Type":"ContainerStarted","Data":"22717b86688d3b65fda3034a026986ff7951351c5ad0908bce14bc7f488cc916"}
Sep 04 23:20:36.763146 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:36.762739    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"
Sep 04 23:20:36.763146 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:36.762763    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6" event={"ID":"259bbe51-fb89-430d-b80f-f77c2af51cc7","Type":"ContainerStarted","Data":"5902b924989835b178080bc0c1f54c8de2e1a38bc3ade3ba8de484e9f026a535"}
Sep 04 23:20:36.783154 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:36.783096    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6" podStartSLOduration=1.78307367 podStartE2EDuration="1.78307367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:20:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:20:36.781159366 +0000 UTC m=+2124.891279952" watchObservedRunningTime="2026-09-04 23:20:36.78307367 +0000 UTC m=+2124.893194256"
Sep 04 23:20:47.771425 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:20:47.771394    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-ktxw6"
Sep 04 23:21:11.216829 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:21:11.216797    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:21:12.906988 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:21:12.906956    2578 scope.go:117] "RemoveContainer" containerID="20615c61222ddb9e9e9366158c59b5af68ebed3563bbc423e7fd0c710a844768"
Sep 04 23:21:12.915905 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:21:12.915882    2578 scope.go:117] "RemoveContainer" containerID="0a98586fd1686e18ece0a5c08686b0c992ccd260d482487795696dead4db9367"
Sep 04 23:21:12.924185 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:21:12.924166    2578 scope.go:117] "RemoveContainer" containerID="375e655a60e1dc8d706e0a2907331971f5a4d777c7ca8d8c4ad24867535b1a52"
Sep 04 23:21:16.922932 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:21:16.922889    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:21:42.597033 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:21:42.596998    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:21:44.503339 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:21:44.503292    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:21:51.473764 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:21:51.473722    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:21:51.579110 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:21:51.579074    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:22:01.504801 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.504761    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"]
Sep 04 23:22:01.505358 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.505339    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerName="istio-proxy"
Sep 04 23:22:01.505358 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.505361    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5425fba2-c853-480c-b386-1ae2b4dbfc40" containerName="istio-proxy"
Sep 04 23:22:01.508449 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.508427    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.511118 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.511098    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 04 23:22:01.511960 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.511943    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-b9gc6\""
Sep 04 23:22:01.512073 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.512022    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 04 23:22:01.521990 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.521962    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"]
Sep 04 23:22:01.600352 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.600319    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m8zr\" (UniqueName: \"kubernetes.io/projected/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-kube-api-access-5m8zr\") pod \"maas-api-657d76c464-ft5zt\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.600516 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.600359    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-metrics-certs\") pod \"maas-api-657d76c464-ft5zt\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.600516 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.600401    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-maas-api-tls\") pod \"maas-api-657d76c464-ft5zt\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.600516 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.600449    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-openshift-service-ca\") pod \"maas-api-657d76c464-ft5zt\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.701450 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.701412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m8zr\" (UniqueName: \"kubernetes.io/projected/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-kube-api-access-5m8zr\") pod \"maas-api-657d76c464-ft5zt\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.701667 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.701459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-metrics-certs\") pod \"maas-api-657d76c464-ft5zt\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.701667 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.701505    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-maas-api-tls\") pod \"maas-api-657d76c464-ft5zt\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.701667 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.701532    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-openshift-service-ca\") pod \"maas-api-657d76c464-ft5zt\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.702230 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.702206    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-openshift-service-ca\") pod \"maas-api-657d76c464-ft5zt\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.703988 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.703964    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-metrics-certs\") pod \"maas-api-657d76c464-ft5zt\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.704074 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.703977    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-maas-api-tls\") pod \"maas-api-657d76c464-ft5zt\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.710813 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.710780    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m8zr\" (UniqueName: \"kubernetes.io/projected/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-kube-api-access-5m8zr\") pod \"maas-api-657d76c464-ft5zt\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:01.819422 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:01.819338    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:02.094827 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:02.094799    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"]
Sep 04 23:22:02.096599 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:22:02.096571    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dbdddd0_7e61_4de4_92ab_cd8950e330e6.slice/crio-fc343536be01f0a2d200c0e7b94bf4e2641f593aa750f71157da19c510047574 WatchSource:0}: Error finding container fc343536be01f0a2d200c0e7b94bf4e2641f593aa750f71157da19c510047574: Status 404 returned error can't find the container with id fc343536be01f0a2d200c0e7b94bf4e2641f593aa750f71157da19c510047574
Sep 04 23:22:02.098074 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:02.098050    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 23:22:02.256090 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:02.256051    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt" event={"ID":"8dbdddd0-7e61-4de4-92ab-cd8950e330e6","Type":"ContainerStarted","Data":"fc343536be01f0a2d200c0e7b94bf4e2641f593aa750f71157da19c510047574"}
Sep 04 23:22:06.281864 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:06.281831    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt" event={"ID":"8dbdddd0-7e61-4de4-92ab-cd8950e330e6","Type":"ContainerStarted","Data":"1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3"}
Sep 04 23:22:06.282273 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:06.281991    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:06.299731 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:06.299678    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt" podStartSLOduration=1.6928675100000001 podStartE2EDuration="5.299664617s" totalImagesPullingTime="3.606797107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:22:06.298320157 +0000 UTC m=+2214.408440746" watchObservedRunningTime="2026-09-04 23:22:06.299664617 +0000 UTC m=+2214.409785196"
Sep 04 23:22:10.945947 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:10.945913    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"]
Sep 04 23:22:10.946344 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:10.946124    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt" podUID="8dbdddd0-7e61-4de4-92ab-cd8950e330e6" containerName="maas-api" containerID="cri-o://1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3" gracePeriod=30
Sep 04 23:22:10.952238 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:10.952209    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:10.979930 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:10.979903    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"]
Sep 04 23:22:10.983321 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:10.983305    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:10.991636 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:10.991608    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"]
Sep 04 23:22:11.092104 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.092068    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e70a043-ff99-46db-a4a7-f192d139196d-openshift-service-ca\") pod \"maas-api-789ccbf67b-srsxz\" (UID: \"6e70a043-ff99-46db-a4a7-f192d139196d\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.092285 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.092135    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e70a043-ff99-46db-a4a7-f192d139196d-metrics-certs\") pod \"maas-api-789ccbf67b-srsxz\" (UID: \"6e70a043-ff99-46db-a4a7-f192d139196d\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.092285 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.092179    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xk8v\" (UniqueName: \"kubernetes.io/projected/6e70a043-ff99-46db-a4a7-f192d139196d-kube-api-access-5xk8v\") pod \"maas-api-789ccbf67b-srsxz\" (UID: \"6e70a043-ff99-46db-a4a7-f192d139196d\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.092285 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.092230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e70a043-ff99-46db-a4a7-f192d139196d-maas-api-tls\") pod \"maas-api-789ccbf67b-srsxz\" (UID: \"6e70a043-ff99-46db-a4a7-f192d139196d\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.192740 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.192710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xk8v\" (UniqueName: \"kubernetes.io/projected/6e70a043-ff99-46db-a4a7-f192d139196d-kube-api-access-5xk8v\") pod \"maas-api-789ccbf67b-srsxz\" (UID: \"6e70a043-ff99-46db-a4a7-f192d139196d\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.192867 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.192777    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e70a043-ff99-46db-a4a7-f192d139196d-maas-api-tls\") pod \"maas-api-789ccbf67b-srsxz\" (UID: \"6e70a043-ff99-46db-a4a7-f192d139196d\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.192922 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.192864    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e70a043-ff99-46db-a4a7-f192d139196d-openshift-service-ca\") pod \"maas-api-789ccbf67b-srsxz\" (UID: \"6e70a043-ff99-46db-a4a7-f192d139196d\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.192967 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.192917    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e70a043-ff99-46db-a4a7-f192d139196d-metrics-certs\") pod \"maas-api-789ccbf67b-srsxz\" (UID: \"6e70a043-ff99-46db-a4a7-f192d139196d\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.193564 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.193537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e70a043-ff99-46db-a4a7-f192d139196d-openshift-service-ca\") pod \"maas-api-789ccbf67b-srsxz\" (UID: \"6e70a043-ff99-46db-a4a7-f192d139196d\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.195139 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.195118    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e70a043-ff99-46db-a4a7-f192d139196d-metrics-certs\") pod \"maas-api-789ccbf67b-srsxz\" (UID: \"6e70a043-ff99-46db-a4a7-f192d139196d\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.195221 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.195170    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e70a043-ff99-46db-a4a7-f192d139196d-maas-api-tls\") pod \"maas-api-789ccbf67b-srsxz\" (UID: \"6e70a043-ff99-46db-a4a7-f192d139196d\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.195771 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.195753    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:11.200538 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.200484    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xk8v\" (UniqueName: \"kubernetes.io/projected/6e70a043-ff99-46db-a4a7-f192d139196d-kube-api-access-5xk8v\") pod \"maas-api-789ccbf67b-srsxz\" (UID: \"6e70a043-ff99-46db-a4a7-f192d139196d\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.294257 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.294226    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-metrics-certs\") pod \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") "
Sep 04 23:22:11.294418 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.294301    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-openshift-service-ca\") pod \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") "
Sep 04 23:22:11.294418 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.294321    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5m8zr\" (UniqueName: \"kubernetes.io/projected/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-kube-api-access-5m8zr\") pod \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") "
Sep 04 23:22:11.294418 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.294350    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-maas-api-tls\") pod \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\" (UID: \"8dbdddd0-7e61-4de4-92ab-cd8950e330e6\") "
Sep 04 23:22:11.294786 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.294762    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8dbdddd0-7e61-4de4-92ab-cd8950e330e6" (UID: "8dbdddd0-7e61-4de4-92ab-cd8950e330e6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:22:11.296597 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.296571    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8dbdddd0-7e61-4de4-92ab-cd8950e330e6" (UID: "8dbdddd0-7e61-4de4-92ab-cd8950e330e6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:22:11.296686 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.296618    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8dbdddd0-7e61-4de4-92ab-cd8950e330e6" (UID: "8dbdddd0-7e61-4de4-92ab-cd8950e330e6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:22:11.296686 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.296635    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-kube-api-access-5m8zr" (OuterVolumeSpecName: "kube-api-access-5m8zr") pod "8dbdddd0-7e61-4de4-92ab-cd8950e330e6" (UID: "8dbdddd0-7e61-4de4-92ab-cd8950e330e6"). InnerVolumeSpecName "kube-api-access-5m8zr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:22:11.311951 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.311914    2578 generic.go:356] "Generic (PLEG): container finished" podID="8dbdddd0-7e61-4de4-92ab-cd8950e330e6" containerID="1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3" exitCode=0
Sep 04 23:22:11.312052 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.311965    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"
Sep 04 23:22:11.312052 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.311999    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt" event={"ID":"8dbdddd0-7e61-4de4-92ab-cd8950e330e6","Type":"ContainerDied","Data":"1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3"}
Sep 04 23:22:11.312136 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.312053    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-657d76c464-ft5zt" event={"ID":"8dbdddd0-7e61-4de4-92ab-cd8950e330e6","Type":"ContainerDied","Data":"fc343536be01f0a2d200c0e7b94bf4e2641f593aa750f71157da19c510047574"}
Sep 04 23:22:11.312136 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.312077    2578 scope.go:117] "RemoveContainer" containerID="1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3"
Sep 04 23:22:11.321530 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.321506    2578 scope.go:117] "RemoveContainer" containerID="1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3"
Sep 04 23:22:11.321837 ip-10-0-128-32 kubenswrapper[2578]: E0904 23:22:11.321817    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3\": container with ID starting with 1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3 not found: ID does not exist" containerID="1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3"
Sep 04 23:22:11.321902 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.321847    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3"} err="failed to get container status \"1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3\": rpc error: code = NotFound desc = could not find container \"1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3\": container with ID starting with 1a26b6e5c92c011bd522a409cf0c1cee0102301c07733550456a4189907b7db3 not found: ID does not exist"
Sep 04 23:22:11.334451 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.334424    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"]
Sep 04 23:22:11.338416 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.338394    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-657d76c464-ft5zt"]
Sep 04 23:22:11.340962 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.340935    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:11.395534 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.395214    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-metrics-certs\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:22:11.395534 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.395250    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-openshift-service-ca\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:22:11.395534 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.395265    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m8zr\" (UniqueName: \"kubernetes.io/projected/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-kube-api-access-5m8zr\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:22:11.395534 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.395280    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbdddd0-7e61-4de4-92ab-cd8950e330e6-maas-api-tls\") on node \"ip-10-0-128-32.ec2.internal\" DevicePath \"\""
Sep 04 23:22:11.611514 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:11.611488    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"]
Sep 04 23:22:11.612811 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:22:11.612781    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e70a043_ff99_46db_a4a7_f192d139196d.slice/crio-fd7a6da2ced173c18e773c22c205ec743d0fc8c13475974147b681c773197ffe WatchSource:0}: Error finding container fd7a6da2ced173c18e773c22c205ec743d0fc8c13475974147b681c773197ffe: Status 404 returned error can't find the container with id fd7a6da2ced173c18e773c22c205ec743d0fc8c13475974147b681c773197ffe
Sep 04 23:22:12.320335 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:12.320246    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz" event={"ID":"6e70a043-ff99-46db-a4a7-f192d139196d","Type":"ContainerStarted","Data":"e9eb567ca705cbf61c7139a2c3ffd4c792470fa2bd0dcc2fe174b06a8fe1a595"}
Sep 04 23:22:12.320335 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:12.320322    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz" event={"ID":"6e70a043-ff99-46db-a4a7-f192d139196d","Type":"ContainerStarted","Data":"fd7a6da2ced173c18e773c22c205ec743d0fc8c13475974147b681c773197ffe"}
Sep 04 23:22:12.320776 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:12.320342    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:12.339193 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:12.339147    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz" podStartSLOduration=1.814696088 podStartE2EDuration="2.339133093s" totalImagesPullingTime="524.437005ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:22:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:22:12.336905735 +0000 UTC m=+2220.447026321" watchObservedRunningTime="2026-09-04 23:22:12.339133093 +0000 UTC m=+2220.449253687"
Sep 04 23:22:12.420975 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:12.420941    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dbdddd0-7e61-4de4-92ab-cd8950e330e6" path="/var/lib/kubelet/pods/8dbdddd0-7e61-4de4-92ab-cd8950e330e6/volumes"
Sep 04 23:22:12.954951 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:12.954919    2578 scope.go:117] "RemoveContainer" containerID="a273e1d901b9189e49e0b7b98b2fba1b91ee31a86cadce3c06d65d13ac68e1be"
Sep 04 23:22:18.332121 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:18.332089    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-srsxz"
Sep 04 23:22:41.899175 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:41.899099    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-4j6mw_69a2aca3-4bef-4b6e-9a7b-af150ef0a71f/manager/0.log"
Sep 04 23:22:42.019682 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:42.019635    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-759c5d6957-4qdnw_924c2f81-6665-473d-9e70-39cc1baee267/copy-manifests/0.log"
Sep 04 23:22:42.025375 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:42.025347    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-759c5d6957-4qdnw_924c2f81-6665-473d-9e70-39cc1baee267/manager/0.log"
Sep 04 23:22:42.803174 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:42.803147    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-789ccbf67b-srsxz_6e70a043-ff99-46db-a4a7-f192d139196d/maas-api/0.log"
Sep 04 23:22:42.916939 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:42.916909    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29809380-p5r44_35cffcac-be02-4cd8-9730-f76b35cf5f4c/cleanup/0.log"
Sep 04 23:22:43.048549 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:43.048510    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29809395-4nj69_eda2463a-78ed-4bdb-a85f-d47ddda0099d/cleanup/0.log"
Sep 04 23:22:43.169010 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:43.168986    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-v4p9p_e94f394a-ce5c-4963-8d87-1b23d7b245fc/postgres/0.log"
Sep 04 23:22:44.144569 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:44.144539    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-nf42w_e9457cc4-c254-4b46-8b34-367d313ce579/manager/0.log"
Sep 04 23:22:44.270045 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:44.270010    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-hhv67_705222be-4838-479d-b8b8-ac69c00e8147/manager/0.log"
Sep 04 23:22:44.541056 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:44.540972    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-ktxw6_259bbe51-fb89-430d-b80f-f77c2af51cc7/manager/0.log"
Sep 04 23:22:45.353324 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:45.353292    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ljj6v_60baba46-17cd-4e43-a0c7-013389ecf4be/discovery/0.log"
Sep 04 23:22:45.490714 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:45.490681    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c4db4689d-7q5ms_0f532f7e-424e-425b-9cdb-031bf519aa94/kube-auth-proxy/0.log"
Sep 04 23:22:45.894062 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:45.894032    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-6vxmn_3646b8ff-9710-4dfd-8702-32d8e3ca23e3/payload-pre-processing/0.log"
Sep 04 23:22:46.038360 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:46.038326    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-2pxrz_337ff73a-57c3-4d64-a0a6-e942e76477d5/payload-processing/0.log"
Sep 04 23:22:46.162483 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:46.162378    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67c5f4dcb-cnpgt_fbbca652-c23c-4eaf-b908-7d724be0ab91/router/0.log"
Sep 04 23:22:46.691022 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:46.690994    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-zw5tn_5f2811b7-ab3e-4c93-ac4e-0f0585c5b2fb/main/0.log"
Sep 04 23:22:47.176429 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:47.176393    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-h5xj7_8efc8e75-8870-45a6-98a0-29c5ef0d8426/main/0.log"
Sep 04 23:22:53.765911 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:53.765877    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:22:53.866335 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:53.866299    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:22:56.576226 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:56.576197    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t4vfv_075c9d8c-8c3d-4509-aca6-fa687f5eb1bb/global-pull-secret-syncer/0.log"
Sep 04 23:22:56.618528 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:56.618498    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log"
Sep 04 23:22:56.695834 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:22:56.695801    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-32.ec2.internal_9dd58ca56a50716b273632f503970b80/haproxy/0.log"
Sep 04 23:23:01.772938 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:01.772853    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-nf42w_e9457cc4-c254-4b46-8b34-367d313ce579/manager/0.log"
Sep 04 23:23:01.802375 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:01.802345    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-hhv67_705222be-4838-479d-b8b8-ac69c00e8147/manager/0.log"
Sep 04 23:23:01.866982 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:01.866947    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-ktxw6_259bbe51-fb89-430d-b80f-f77c2af51cc7/manager/0.log"
Sep 04 23:23:03.358868 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:03.358838    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d141747-26de-41f3-80b8-1544d1bc444b/alertmanager/0.log"
Sep 04 23:23:03.379083 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:03.379051    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d141747-26de-41f3-80b8-1544d1bc444b/config-reloader/0.log"
Sep 04 23:23:03.397878 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:03.397849    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d141747-26de-41f3-80b8-1544d1bc444b/kube-rbac-proxy-web/0.log"
Sep 04 23:23:03.415841 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:03.415813    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d141747-26de-41f3-80b8-1544d1bc444b/kube-rbac-proxy/0.log"
Sep 04 23:23:03.433240 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:03.433210    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d141747-26de-41f3-80b8-1544d1bc444b/kube-rbac-proxy-metric/0.log"
Sep 04 23:23:03.452755 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:03.452723    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d141747-26de-41f3-80b8-1544d1bc444b/prom-label-proxy/0.log"
Sep 04 23:23:03.473441 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:03.473414    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d141747-26de-41f3-80b8-1544d1bc444b/init-config-reloader/0.log"
Sep 04 23:23:03.540934 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:03.540907    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-2vhtw_e05e4642-b09c-4900-a384-f433d0f6188e/kube-state-metrics/0.log"
Sep 04 23:23:03.564108 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:03.564081    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-2vhtw_e05e4642-b09c-4900-a384-f433d0f6188e/kube-rbac-proxy-main/0.log"
Sep 04 23:23:03.582054 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:03.582030    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-2vhtw_e05e4642-b09c-4900-a384-f433d0f6188e/kube-rbac-proxy-self/0.log"
Sep 04 23:23:03.994064 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:03.993984    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p6g4x_1c3a1a96-67cc-4068-8bcd-b949c651fd26/node-exporter/0.log"
Sep 04 23:23:04.012700 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:04.012665    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p6g4x_1c3a1a96-67cc-4068-8bcd-b949c651fd26/kube-rbac-proxy/0.log"
Sep 04 23:23:04.030067 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:04.030045    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p6g4x_1c3a1a96-67cc-4068-8bcd-b949c651fd26/init-textfile/0.log"
Sep 04 23:23:04.299150 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:04.299076    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-dpv8r_eb50d22d-682c-47cd-a778-c7c6af5cc95e/prometheus-operator/0.log"
Sep 04 23:23:04.319281 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:04.319234    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-dpv8r_eb50d22d-682c-47cd-a778-c7c6af5cc95e/kube-rbac-proxy/0.log"
Sep 04 23:23:05.537618 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:05.537579    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6748j_b3de408b-3ab1-4bf9-9898-2ca1b2ccd8ef/konnectivity-agent/0.log" limit=1024
Sep 04 23:23:05.818434 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:05.818326    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-ckz9h_d5878698-1651-4f18-b257-e5f0fa24e1d0/networking-console-plugin/0.log"
Sep 04 23:23:06.354150 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:06.354088    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/1.log"
Sep 04 23:23:06.359579 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:06.359549    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-58vxq_6c89c14f-e6b5-4754-9182-64a5858d0e2e/console-operator/2.log"
Sep 04 23:23:08.115791 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.115752    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"]
Sep 04 23:23:08.116431 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.116126    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dbdddd0-7e61-4de4-92ab-cd8950e330e6" containerName="maas-api"
Sep 04 23:23:08.116431 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.116145    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dbdddd0-7e61-4de4-92ab-cd8950e330e6" containerName="maas-api"
Sep 04 23:23:08.116431 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.116221    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dbdddd0-7e61-4de4-92ab-cd8950e330e6" containerName="maas-api"
Sep 04 23:23:08.119376 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.119359    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.122122 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.122097    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vpm2f\"/\"kube-root-ca.crt\""
Sep 04 23:23:08.122258 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.122237    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vpm2f\"/\"default-dockercfg-s42c6\""
Sep 04 23:23:08.123253 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.123168    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vpm2f\"/\"openshift-service-ca.crt\""
Sep 04 23:23:08.129042 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.129020    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"]
Sep 04 23:23:08.313815 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.313775    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-proc\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.314006 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.313849    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w49gg\" (UniqueName: \"kubernetes.io/projected/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-kube-api-access-w49gg\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.314006 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.313897    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-sys\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.314006 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.313934    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-podres\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.314006 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.313977    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-lib-modules\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.414615 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.414519    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-proc\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.414615 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.414558    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w49gg\" (UniqueName: \"kubernetes.io/projected/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-kube-api-access-w49gg\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.414615 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.414592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-sys\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.414900 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.414634    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-podres\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.414900 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.414668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-proc\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.414900 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.414685    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-lib-modules\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.414900 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.414747    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-sys\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.414900 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.414807    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-lib-modules\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.414900 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.414813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-podres\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.422447 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.422423    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w49gg\" (UniqueName: \"kubernetes.io/projected/b0364b28-ff46-43b8-88b0-1aa9cf8f82f0-kube-api-access-w49gg\") pod \"perf-node-gather-daemonset-lg8f9\" (UID: \"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.431309 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.431291    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:08.701195 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.701167    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"]
Sep 04 23:23:08.702636 ip-10-0-128-32 kubenswrapper[2578]: W0904 23:23:08.702607    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb0364b28_ff46_43b8_88b0_1aa9cf8f82f0.slice/crio-407af81e653c7f022d4087d8d2d0776312161f2932c9a8b78bb7a3adeb6747c5 WatchSource:0}: Error finding container 407af81e653c7f022d4087d8d2d0776312161f2932c9a8b78bb7a3adeb6747c5: Status 404 returned error can't find the container with id 407af81e653c7f022d4087d8d2d0776312161f2932c9a8b78bb7a3adeb6747c5
Sep 04 23:23:08.791756 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.791730    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mpx5p_8242b073-28b9-46e8-8783-9aa8871bcf2e/dns/0.log"
Sep 04 23:23:08.809059 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.809022    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mpx5p_8242b073-28b9-46e8-8783-9aa8871bcf2e/kube-rbac-proxy/0.log"
Sep 04 23:23:08.855612 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:08.855591    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k7nf5_7655d1a4-b050-4eea-b049-7c6899295630/dns-node-resolver/0.log"
Sep 04 23:23:09.407115 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:09.407041    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4qbrc_407039b1-b5f0-4538-8f92-f277e742b9d6/node-ca/0.log"
Sep 04 23:23:09.648069 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:09.648028    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9" event={"ID":"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0","Type":"ContainerStarted","Data":"25a2ff9271d4aa6276a0c138d855a202c12cd8e779d5a09015a1424d201d8163"}
Sep 04 23:23:09.648253 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:09.648092    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9" event={"ID":"b0364b28-ff46-43b8-88b0-1aa9cf8f82f0","Type":"ContainerStarted","Data":"407af81e653c7f022d4087d8d2d0776312161f2932c9a8b78bb7a3adeb6747c5"}
Sep 04 23:23:09.648302 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:09.648259    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:09.664399 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:09.664289    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9" podStartSLOduration=1.6642725010000001 podStartE2EDuration="1.664272501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:23:09.662740762 +0000 UTC m=+2277.772861349" watchObservedRunningTime="2026-09-04 23:23:09.664272501 +0000 UTC m=+2277.774393124"
Sep 04 23:23:10.578107 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:10.578071    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ljj6v_60baba46-17cd-4e43-a0c7-013389ecf4be/discovery/0.log"
Sep 04 23:23:10.595159 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:10.595136    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c4db4689d-7q5ms_0f532f7e-424e-425b-9cdb-031bf519aa94/kube-auth-proxy/0.log"
Sep 04 23:23:10.681452 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:10.681422    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-6vxmn_3646b8ff-9710-4dfd-8702-32d8e3ca23e3/payload-pre-processing/0.log"
Sep 04 23:23:10.711611 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:10.711585    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-2pxrz_337ff73a-57c3-4d64-a0a6-e942e76477d5/payload-processing/0.log"
Sep 04 23:23:10.731157 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:10.731132    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67c5f4dcb-cnpgt_fbbca652-c23c-4eaf-b908-7d724be0ab91/router/0.log"
Sep 04 23:23:11.559075 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:11.559047    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hx25v_565701c0-684e-4fc0-9bc5-2ebce20c2100/serve-healthcheck-canary/0.log"
Sep 04 23:23:12.080418 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:12.080388    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8xsql_8cd78b64-4ed2-4d53-87af-2fa41d0763ae/kube-rbac-proxy/0.log"
Sep 04 23:23:12.100597 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:12.100576    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8xsql_8cd78b64-4ed2-4d53-87af-2fa41d0763ae/exporter/0.log"
Sep 04 23:23:12.118855 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:12.118821    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8xsql_8cd78b64-4ed2-4d53-87af-2fa41d0763ae/extractor/0.log"
Sep 04 23:23:12.980844 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:12.980808    2578 scope.go:117] "RemoveContainer" containerID="0f48c9fda25d1fef0da825bc58841743d4fb9489f2e6ee171a66b0d96372d09c"
Sep 04 23:23:12.989779 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:12.989755    2578 scope.go:117] "RemoveContainer" containerID="61ddf4e44158df3ae296495d671e6561a6f00dc544d7cb9aa17e773e9e427a1c"
Sep 04 23:23:14.583690 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:14.583651    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-4j6mw_69a2aca3-4bef-4b6e-9a7b-af150ef0a71f/manager/0.log"
Sep 04 23:23:14.606286 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:14.606255    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-759c5d6957-4qdnw_924c2f81-6665-473d-9e70-39cc1baee267/manager/0.log"
Sep 04 23:23:14.624046 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:14.624018    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-759c5d6957-4qdnw_924c2f81-6665-473d-9e70-39cc1baee267/copy-manifests/0.log"
Sep 04 23:23:15.664337 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:15.664309    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-lg8f9"
Sep 04 23:23:15.902489 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:15.902462    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-grp6h_bb7edda7-d710-43d2-a0b6-d9011c1bfb22/openshift-lws-operator/0.log"
Sep 04 23:23:17.125092 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:17.125059    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ljj6v_60baba46-17cd-4e43-a0c7-013389ecf4be/discovery/0.log"
Sep 04 23:23:17.139429 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:17.139407    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c4db4689d-7q5ms_0f532f7e-424e-425b-9cdb-031bf519aa94/kube-auth-proxy/0.log"
Sep 04 23:23:17.212446 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:17.212409    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-6vxmn_3646b8ff-9710-4dfd-8702-32d8e3ca23e3/payload-pre-processing/0.log"
Sep 04 23:23:17.236509 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:17.236478    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-2pxrz_337ff73a-57c3-4d64-a0a6-e942e76477d5/payload-processing/0.log"
Sep 04 23:23:17.248680 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:17.248653    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67c5f4dcb-cnpgt_fbbca652-c23c-4eaf-b908-7d724be0ab91/router/0.log"
Sep 04 23:23:19.099047 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:19.099014    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ljj6v_60baba46-17cd-4e43-a0c7-013389ecf4be/discovery/0.log"
Sep 04 23:23:19.115187 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:19.115165    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c4db4689d-7q5ms_0f532f7e-424e-425b-9cdb-031bf519aa94/kube-auth-proxy/0.log"
Sep 04 23:23:19.191291 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:19.191253    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-6vxmn_3646b8ff-9710-4dfd-8702-32d8e3ca23e3/payload-pre-processing/0.log"
Sep 04 23:23:19.216630 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:19.216604    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-2pxrz_337ff73a-57c3-4d64-a0a6-e942e76477d5/payload-processing/0.log"
Sep 04 23:23:19.233060 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:19.233038    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67c5f4dcb-cnpgt_fbbca652-c23c-4eaf-b908-7d724be0ab91/router/0.log"
Sep 04 23:23:22.508456 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:22.508412    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9vmrt_aa4adc0d-b12a-4f86-addd-5c39a9fc1255/kube-multus-additional-cni-plugins/0.log"
Sep 04 23:23:22.525313 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:22.525289    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9vmrt_aa4adc0d-b12a-4f86-addd-5c39a9fc1255/egress-router-binary-copy/0.log"
Sep 04 23:23:22.547866 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:22.547845    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9vmrt_aa4adc0d-b12a-4f86-addd-5c39a9fc1255/cni-plugins/0.log"
Sep 04 23:23:22.565612 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:22.565592    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9vmrt_aa4adc0d-b12a-4f86-addd-5c39a9fc1255/bond-cni-plugin/0.log"
Sep 04 23:23:22.582276 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:22.582256    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9vmrt_aa4adc0d-b12a-4f86-addd-5c39a9fc1255/routeoverride-cni/0.log"
Sep 04 23:23:22.600361 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:22.600343    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9vmrt_aa4adc0d-b12a-4f86-addd-5c39a9fc1255/whereabouts-cni-bincopy/0.log"
Sep 04 23:23:22.617905 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:22.617886    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9vmrt_aa4adc0d-b12a-4f86-addd-5c39a9fc1255/whereabouts-cni/0.log"
Sep 04 23:23:22.939865 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:22.939817    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qdvpv_0d6465b5-2c07-420d-b17d-a9f524a07220/kube-multus/0.log"
Sep 04 23:23:23.029949 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:23.029900    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9dqmt_a7ec67b5-ff79-49d1-b063-8cd7c4a56c84/network-metrics-daemon/0.log"
Sep 04 23:23:23.044446 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:23.044424    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9dqmt_a7ec67b5-ff79-49d1-b063-8cd7c4a56c84/kube-rbac-proxy/0.log"
Sep 04 23:23:24.276973 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:24.276940    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-znc4n_6e5ea01c-a8df-4300-b94b-d4b9b0828846/ovn-controller/0.log"
Sep 04 23:23:24.290770 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:24.290739    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-znc4n_6e5ea01c-a8df-4300-b94b-d4b9b0828846/ovn-acl-logging/0.log"
Sep 04 23:23:24.312928 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:24.312890    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-znc4n_6e5ea01c-a8df-4300-b94b-d4b9b0828846/kube-rbac-proxy-node/0.log"
Sep 04 23:23:24.330714 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:24.330688    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-znc4n_6e5ea01c-a8df-4300-b94b-d4b9b0828846/kube-rbac-proxy-ovn-metrics/0.log"
Sep 04 23:23:24.347129 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:24.347105    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-znc4n_6e5ea01c-a8df-4300-b94b-d4b9b0828846/northd/0.log"
Sep 04 23:23:24.365185 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:24.365166    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-znc4n_6e5ea01c-a8df-4300-b94b-d4b9b0828846/nbdb/0.log"
Sep 04 23:23:24.383834 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:24.383817    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-znc4n_6e5ea01c-a8df-4300-b94b-d4b9b0828846/sbdb/0.log"
Sep 04 23:23:24.439379 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:24.439354    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-znc4n_6e5ea01c-a8df-4300-b94b-d4b9b0828846/ovnkube-controller/0.log"
Sep 04 23:23:25.526967 ip-10-0-128-32 kubenswrapper[2578]: I0904 23:23:25.526943    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-77nr4_9c9d02b6-4018-4f24-9900-df3d1eadbc59/network-check-target-container/0.log"