Jul 10 03:32:18.382928 ip-10-0-140-167 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 03:32:18.823869 ip-10-0-140-167 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 03:32:18.823869 ip-10-0-140-167 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 03:32:18.823869 ip-10-0-140-167 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 03:32:18.823869 ip-10-0-140-167 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 03:32:18.823869 ip-10-0-140-167 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 03:32:18.826779 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.826660    2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 03:32:18.831586 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831570    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 03:32:18.831586 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831586    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831591    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831594    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831597    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831600    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831603    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831609    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831613    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831616    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831618    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831621    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831624    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831626    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831629    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831632    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831634    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831637    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831640    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831642    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 03:32:18.831668 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831656    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831660    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831663    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831665    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831668    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831671    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831674    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831677    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831680    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831682    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831685    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831687    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831690    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831695    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831699    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831702    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831705    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831708    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831711    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 03:32:18.832121 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831713    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831716    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831719    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831721    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831724    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831726    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831729    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831731    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831734    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831737    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831739    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831742    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831744    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831747    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831749    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831753    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831756    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831760    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831762    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831765    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 03:32:18.832567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831768    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831770    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831773    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831777    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831780    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831783    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831786    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831789    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831792    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831794    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831797    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831799    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831802    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831805    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831807    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831810    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831812    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831815    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831817    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 03:32:18.833066 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831820    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831823    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831825    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831828    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831830    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831833    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831835    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831838    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831841    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831844    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831847    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831849    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831852    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831855    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831857    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831860    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831862    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831865    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831867    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831870    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 03:32:18.833518 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.831872    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 03:32:18.834257 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834244    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 03:32:18.834257 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834256    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834260    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834263    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834266    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834270    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834273    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834276    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834279    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834281    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834284    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834286    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834289    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834293    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834297    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834300    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834302    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834305    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834307    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834310    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 03:32:18.834312 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834313    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834316    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834319    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834321    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834324    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834327    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834330    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834332    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834335    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834337    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834340    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834342    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834345    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834347    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834350    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834353    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834355    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834357    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834361    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834363    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 03:32:18.834813 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834366    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834368    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834371    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834373    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834376    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834378    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834380    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834383    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834386    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834388    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834391    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834394    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834396    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834399    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834402    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834405    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834408    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834410    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834413    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834415    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 03:32:18.835318 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834418    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834420    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834423    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834426    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834430    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834433    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834436    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834439    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834442    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834445    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834447    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834451    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834455    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834458    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834461    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834470    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834474    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834477    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834479    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 03:32:18.835819 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834482    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834484    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834487    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834490    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834492    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834495    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834497    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834503    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834505    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834508    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834511    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834514    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834516    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834519    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834521    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834524    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834526    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834529    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834532    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 03:32:18.836285 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834534    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834933    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834938    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834941    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834944    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834949    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834953    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834956    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834959    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834962    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834965    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834968    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834971    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834974    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834976    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834980    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834983    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834986    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834988    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 03:32:18.836754 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834991    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834995    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.834999    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835002    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835004    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835007    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835011    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835014    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835016    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835019    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835022    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835024    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835027    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835030    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835032    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835037    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835040    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835043    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835046    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835049    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 03:32:18.837225 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835052    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835055    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835058    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835061    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835063    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835067    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835070    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835073    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835075    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835078    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835080    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835084    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835086    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835089    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835091    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835094    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835096    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835098    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835101    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835103    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 03:32:18.837759 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835106    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835108    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835111    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835113    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835116    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835118    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835121    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835123    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835126    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835128    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835131    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835133    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835136    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835139    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835142    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835144    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835147    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835150    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835153    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835155    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 03:32:18.838240 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835158    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835161    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835163    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835166    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835168    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835171    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835173    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835176    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835178    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835181    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835183    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835186    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835189    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835192    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835194    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835197    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835200    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835202    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835205    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 03:32:18.838745 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835207    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835210    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835288    2561 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835298    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835307    2561 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835314    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835319    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835323    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835327    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835332    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835336    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835339    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835342    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835346    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835349    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835352    2561 flags.go:64] FLAG: --cgroup-root=""
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835355    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835358    2561 flags.go:64] FLAG: --client-ca-file=""
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835360    2561 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835363    2561 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835368    2561 flags.go:64] FLAG: --cluster-domain=""
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835371    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835374    2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835377    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 03:32:18.839233 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835380    2561 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835384    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835387    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835390    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835394    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835397    2561 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835400    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835403    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835406    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835409    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835413    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835416    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835419    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835421    2561 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835424    2561 flags.go:64] FLAG: --enable-server="true"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835428    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835435    2561 flags.go:64] FLAG: --event-burst="100"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835438    2561 flags.go:64] FLAG: --event-qps="50"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835441    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835444    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835447    2561 flags.go:64] FLAG: --eviction-hard=""
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835451    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835454    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835457    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835460    2561 flags.go:64] FLAG: --eviction-soft=""
Jul 10 03:32:18.839839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835463    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835465    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835468    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835471    2561 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835474    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835477    2561 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835479    2561 flags.go:64] FLAG: --feature-gates=""
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835483    2561 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835486    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835489    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835492    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835495    2561 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835498    2561 flags.go:64] FLAG: --help="false"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835501    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-140-167.ec2.internal"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835504    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835507    2561 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835510    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835515    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835518    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835521    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835524    2561 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835527    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835529    2561 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 03:32:18.840479 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835535    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835538    2561 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835541    2561 flags.go:64] FLAG: --kube-reserved=""
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835544    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835548    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835551    2561 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835553    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835556    2561 flags.go:64] FLAG: --lock-file=""
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835559    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835562    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835565    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835570    2561 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835573    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835576    2561 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835579    2561 flags.go:64] FLAG: --logging-format="text"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835581    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835585    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835587    2561 flags.go:64] FLAG: --manifest-url=""
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835590    2561 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835594    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835597    2561 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835601    2561 flags.go:64] FLAG: --max-pods="110"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835604    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835607    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835610    2561 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 03:32:18.841096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835613    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835617    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835620    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835623    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835630    2561 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835633    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835636    2561 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835641    2561 flags.go:64] FLAG: --pod-cidr=""
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835656    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835662    2561 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835665    2561 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835678    2561 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835682    2561 flags.go:64] FLAG: --port="10250"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835685    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835688    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-033ab4615fd96241a"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835691    2561 flags.go:64] FLAG: --qos-reserved=""
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835694    2561 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835697    2561 flags.go:64] FLAG: --register-node="true"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835700    2561 flags.go:64] FLAG: --register-with-taints=""
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835704    2561 flags.go:64] FLAG: --registry-burst="10"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835706    2561 flags.go:64] FLAG: --registry-qps="5"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835709    2561 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835712    2561 flags.go:64] FLAG: --reserved-memory=""
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835716    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835719    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 03:32:18.841730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835721    2561 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835724    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835727    2561 flags.go:64] FLAG: --runonce="false"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835730    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835733    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835736    2561 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835738    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835741    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835744    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835748    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835752    2561 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835755    2561 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835758    2561 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835760    2561 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835763    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835767    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835771    2561 flags.go:64] FLAG: --system-cgroups=""
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835773    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835779    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835782    2561 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835785    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835789    2561 flags.go:64] FLAG: --tls-min-version=""
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835792    2561 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835795    2561 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835798    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 03:32:18.842366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835800    2561 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835803    2561 flags.go:64] FLAG: --v="2"
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835807    2561 flags.go:64] FLAG: --version="false"
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835811    2561 flags.go:64] FLAG: --vmodule=""
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835814    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.835818    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835938    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835941    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835945    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835948    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835950    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835953    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835956    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835959    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835961    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835964    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835970    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835974    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835977    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835980    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835983    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 03:32:18.843114 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835986    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835990    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835993    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835996    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.835999    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836002    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836004    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836007    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836010    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836013    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836016    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836018    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836021    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836023    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836026    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836029    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836031    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836033    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836036    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836039    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 03:32:18.843613 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836042    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836044    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836047    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836049    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836052    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836054    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836057    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836060    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836063    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836066    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836068    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836071    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836074    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836077    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836080    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836083    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836086    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836088    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836091    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836093    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 03:32:18.844209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836096    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836098    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836101    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836103    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836106    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836108    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836111    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836115    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836118    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836121    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836124    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836127    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836129    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836132    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836135    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836138    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836141    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836144    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836147    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 03:32:18.844715 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836151    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836154    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836156    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836160    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836164    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836167    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836171    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836174    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836176    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836179    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836182    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836184    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836187    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836190    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836192    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836195    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836197    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836200    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836202    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 03:32:18.845183 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836205    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836207    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836210    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836212    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836215    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.836217    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.836940    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.844791    2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.844806    2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.844826    2561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.844832    2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844898    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844904    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844909    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 03:32:18.845659 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844912    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844914    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844918    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844922    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844926    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844929    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844932    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844934    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844938    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844948    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844951    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844954    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844958    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844960    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844963    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844966    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844969    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844972    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844974    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 03:32:18.846020 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844977    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844980    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844983    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844985    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844988    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844990    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844994    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.844997    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845000    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845003    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845005    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845008    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845011    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845013    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845016    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845018    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845021    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845023    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845026    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845029    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 03:32:18.846459 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845031    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845034    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845036    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845045    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845047    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845050    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845052    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845055    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845057    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845060    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845062    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845065    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845067    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845070    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845072    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845075    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845078    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845081    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845084    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845087    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 03:32:18.847027 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845089    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845092    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845095    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845097    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845100    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845102    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845105    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845107    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845110    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845112    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845116    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845120    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845122    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845125    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845128    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845131    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845139    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845142    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845144    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 03:32:18.847500 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845147    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845150    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845153    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845155    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845158    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845160    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845163    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845166    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845168    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845171    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845173    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845176    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845179    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845182    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845184    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845187    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845190    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 03:32:18.848213 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845193    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.845198    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845357    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845363    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845366    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845369    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845372    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845375    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845378    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845381    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845383    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845386    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845389    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845397    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845400    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 03:32:18.848731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845403    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845405    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845408    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845411    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845413    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845416    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845418    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845421    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845423    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845426    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845429    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845431    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845434    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845436    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845439    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845441    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845444    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845446    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845449    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845451    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 03:32:18.849102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845454    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845457    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845459    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845462    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845465    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845467    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845470    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845472    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845475    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845477    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845480    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845489    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845492    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845495    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845498    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845500    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845503    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845505    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845508    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845510    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 03:32:18.849567 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845513    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845516    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845519    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845522    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845525    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845529    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845532    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845535    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845538    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845541    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845544    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845546    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845549    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845551    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845554    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845556    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845560    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845564    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845567    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 03:32:18.850077 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845569    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845572    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845575    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845578    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845580    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845589    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845591    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845594    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845597    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845599    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845602    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845604    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845607    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845610    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845612    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845615    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845618    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845621    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845624    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845628    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 03:32:18.850642 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845631    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 03:32:18.851217 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845634    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 03:32:18.851217 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845636    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 03:32:18.851217 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845639    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 03:32:18.851217 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845641    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 03:32:18.851217 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845644    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 03:32:18.851217 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:18.845659    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 03:32:18.851217 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.845664    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 03:32:18.851217 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.845782    2561 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 03:32:18.851217 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.850265    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 03:32:18.852255 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.852238    2561 server.go:1019] "Starting client certificate rotation"
Jul 10 03:32:18.852362 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.852345    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 03:32:18.852391 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.852379    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 03:32:18.855165 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.855150    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 03:32:18.856489 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.856468    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 03:32:18.866157 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.866137    2561 log.go:25] "Validated CRI v1 runtime API"
Jul 10 03:32:18.869691 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.869674    2561 log.go:25] "Validated CRI v1 image API"
Jul 10 03:32:18.870513 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.870486    2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 03:32:18.873214 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.873188    2561 fs.go:135] Filesystem UUIDs: map[41a451d4-7dfb-4e4f-b836-842e0bafcd86:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8bca4476-4a9f-4595-ab0e-b135238e84a0:/dev/nvme0n1p3]
Jul 10 03:32:18.873282 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.873212    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 03:32:18.879611 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.879592    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 03:32:18.879796 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.879681    2561 manager.go:216] Machine: {Timestamp:2026-07-10 03:32:18.878216528 +0000 UTC m=+0.369212392 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091919 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec248ca1dc6346fbb94021e500e9d99f SystemUUID:ec248ca1-dc63-46fb-b940-21e500e9d99f BootID:cf4fb1a2-2ad7-4f0a-a317-e78f727dfac2 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f5:cb:4d:fa:7d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f5:cb:4d:fa:7d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:4a:03:db:ca:b7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 03:32:18.879796 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.879793    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 03:32:18.879898 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.879881    2561 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 03:32:18.883396 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.883365    2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 03:32:18.883569 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.883399    2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-167.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 03:32:18.884327 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.884317    2561 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 03:32:18.884364 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.884330    2561 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 03:32:18.884364 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.884338    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 03:32:18.885972 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.885962    2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 03:32:18.887853 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.887843    2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 03:32:18.892587 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.892575    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 03:32:18.892740 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.892730    2561 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 03:32:18.892783 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.892762    2561 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 03:32:18.892783 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.892774    2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 03:32:18.892860 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.892793    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 03:32:18.892860 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.892804    2561 kubelet.go:398] "Adding apiserver pod source"
Jul 10 03:32:18.892860 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.892814    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 03:32:18.893166 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.893154    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 03:32:18.893222 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.893171    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 03:32:18.895507 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.895489    2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 03:32:18.899113 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.899051    2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 03:32:18.899205 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.899133    2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 03:32:18.900769 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.900751    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 03:32:18.900855 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.900774    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 03:32:18.900855 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.900784    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 03:32:18.900855 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.900792    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 03:32:18.900855 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.900804    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 03:32:18.900855 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.900812    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 03:32:18.900855 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.900825    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 03:32:18.900855 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.900833    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 03:32:18.900855 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.900844    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 03:32:18.900855 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.900854    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 03:32:18.901103 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.900867    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 03:32:18.901103 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.900875    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 03:32:18.901721 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.901710    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 03:32:18.901776 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.901724    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 03:32:18.904829 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.904814    2561 server.go:1268] "Started kubelet"
Jul 10 03:32:18.904952 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.904902    2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 03:32:18.905035 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.904980    2561 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 03:32:18.905083 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.905059    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 03:32:18.905746 ip-10-0-140-167 systemd[1]: Started Kubernetes Kubelet.
Jul 10 03:32:18.906474 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.906444    2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-167.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 03:32:18.906610 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.906594    2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 03:32:18.907122 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.907101    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-167.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 03:32:18.907122 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.907103    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 03:32:18.907567 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.907551    2561 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 03:32:18.911330 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.911313    2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 03:32:18.914137 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.914118    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 03:32:18.914243 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.913001    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-167.ec2.internal.18c0d02bc88111b7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-167.ec2.internal,UID:ip-10-0-140-167.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-167.ec2.internal,},FirstTimestamp:2026-07-10 03:32:18.904789431 +0000 UTC m=+0.395785296,LastTimestamp:2026-07-10 03:32:18.904789431 +0000 UTC m=+0.395785296,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-167.ec2.internal,}"
Jul 10 03:32:18.914243 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.914193    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 03:32:18.914243 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.914214    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:18.914870 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.914848    2561 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 03:32:18.914870 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.914870    2561 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 03:32:18.915011 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.914991    2561 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 03:32:18.915011 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.915002    2561 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 03:32:18.915112 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.915029    2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 03:32:18.915112 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.915051    2561 factory.go:55] Registering systemd factory
Jul 10 03:32:18.915112 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.915066    2561 factory.go:223] Registration of the systemd container factory successfully
Jul 10 03:32:18.915438 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.915420    2561 factory.go:153] Registering CRI-O factory
Jul 10 03:32:18.915438 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.915438    2561 factory.go:223] Registration of the crio container factory successfully
Jul 10 03:32:18.915560 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.915490    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 03:32:18.915560 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.915514    2561 factory.go:103] Registering Raw factory
Jul 10 03:32:18.915560 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.915529    2561 manager.go:1196] Started watching for new ooms in manager
Jul 10 03:32:18.915986 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.915973    2561 manager.go:349] Starting recovery of all containers
Jul 10 03:32:18.925970 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.925943    2561 manager.go:354] Recovery completed
Jul 10 03:32:18.927803 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.927747    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 03:32:18.927884 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.927708    2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-167.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 03:32:18.933265 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.933250    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:18.935468 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.935447    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:18.935551 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.935480    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:18.935551 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.935491    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:18.936106 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.936092    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 03:32:18.936106 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.936106    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 03:32:18.936198 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.936122    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 03:32:18.936198 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.936125    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:18.936831 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.936816    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:18.936871 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.936841    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:18.936871 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.936850    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:18.938242 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.938230    2561 policy_none.go:49] "None policy: Start"
Jul 10 03:32:18.938278 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.938245    2561 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 03:32:18.938278 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.938255    2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 03:32:18.940253 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.940238    2561 policy_none.go:47] "Start"
Jul 10 03:32:18.949739 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.949631    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-167.ec2.internal.18c0d02bca552edf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-167.ec2.internal,UID:ip-10-0-140-167.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-167.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-167.ec2.internal,},FirstTimestamp:2026-07-10 03:32:18.935467743 +0000 UTC m=+0.426463610,LastTimestamp:2026-07-10 03:32:18.935467743 +0000 UTC m=+0.426463610,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-167.ec2.internal,}"
Jul 10 03:32:18.954306 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.954288    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bzntp"
Jul 10 03:32:18.959644 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.959460    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-167.ec2.internal.18c0d02bca55736c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-167.ec2.internal,UID:ip-10-0-140-167.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-167.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-167.ec2.internal,},FirstTimestamp:2026-07-10 03:32:18.935485292 +0000 UTC m=+0.426481156,LastTimestamp:2026-07-10 03:32:18.935485292 +0000 UTC m=+0.426481156,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-167.ec2.internal,}"
Jul 10 03:32:18.963875 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.963854    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bzntp"
Jul 10 03:32:18.970335 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.970271    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-167.ec2.internal.18c0d02bca559980  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-167.ec2.internal,UID:ip-10-0-140-167.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-167.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-167.ec2.internal,},FirstTimestamp:2026-07-10 03:32:18.93549504 +0000 UTC m=+0.426490903,LastTimestamp:2026-07-10 03:32:18.93549504 +0000 UTC m=+0.426490903,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-167.ec2.internal,}"
Jul 10 03:32:18.985319 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.985300    2561 manager.go:337] "Starting Device Plugin manager"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.985335    2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.985346    2561 server.go:87] "Starting device plugin registration server"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.985600    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.985611    2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.985780    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.985843    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.985852    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.986078    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.986475    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.986531    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.987407    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.987441    2561 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.987459    2561 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:18.987488    2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 03:32:18.999444 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:18.990084    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 03:32:19.086229 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.086139    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:19.087225 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.087197    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:19.087309 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.087239    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:19.087309 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.087260    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:19.087309 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.087284    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.087567 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.087550    2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal"]
Jul 10 03:32:19.087632 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.087622    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:19.088423 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.088410    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:19.088490 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.088435    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:19.088490 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.088448    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:19.090851 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.090840    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:19.090990 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.090977    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.091034 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.091007    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:19.092436 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.092417    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:19.092546 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.092442    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:19.092546 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.092450    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:19.092546 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.092468    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:19.092546 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.092483    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:19.092546 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.092484    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:19.094964 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.094945    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.095048 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.094980    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:19.095725 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.095705    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:19.095828 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.095735    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:19.095828 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.095745    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:19.097358 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.097344    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.097425 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:19.097364    2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-167.ec2.internal\": node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:19.116644 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.116609    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9c6f7f9103477741d51b536b17fca268-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal\" (UID: \"9c6f7f9103477741d51b536b17fca268\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.116644 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.116641    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c6f7f9103477741d51b536b17fca268-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal\" (UID: \"9c6f7f9103477741d51b536b17fca268\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.116864 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.116712    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d6cf1273741a1c23d1af7ea6ad5bb863-config\") pod \"kube-apiserver-proxy-ip-10-0-140-167.ec2.internal\" (UID: \"d6cf1273741a1c23d1af7ea6ad5bb863\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.123746 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:19.123722    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:19.127758 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:19.127736    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-167.ec2.internal\" not found" node="ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.132365 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:19.132347    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-167.ec2.internal\" not found" node="ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.217323 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.217295    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9c6f7f9103477741d51b536b17fca268-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal\" (UID: \"9c6f7f9103477741d51b536b17fca268\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.217449 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.217327    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c6f7f9103477741d51b536b17fca268-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal\" (UID: \"9c6f7f9103477741d51b536b17fca268\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.217449 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.217352    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d6cf1273741a1c23d1af7ea6ad5bb863-config\") pod \"kube-apiserver-proxy-ip-10-0-140-167.ec2.internal\" (UID: \"d6cf1273741a1c23d1af7ea6ad5bb863\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.217449 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.217400    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d6cf1273741a1c23d1af7ea6ad5bb863-config\") pod \"kube-apiserver-proxy-ip-10-0-140-167.ec2.internal\" (UID: \"d6cf1273741a1c23d1af7ea6ad5bb863\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.217449 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.217404    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c6f7f9103477741d51b536b17fca268-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal\" (UID: \"9c6f7f9103477741d51b536b17fca268\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.217449 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.217400    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9c6f7f9103477741d51b536b17fca268-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal\" (UID: \"9c6f7f9103477741d51b536b17fca268\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.224387 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:19.224363    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:19.324491 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:19.324460    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:19.425389 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:19.425318    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:19.430498 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.430477    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.437115 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.437090    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:19.526043 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:19.526002    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:19.626498 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:19.626471    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:19.726907 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:19.726834    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:19.728944 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.728928    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 03:32:19.827762 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:19.827723    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:19.851899 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.851872    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 03:32:19.852072 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.852050    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 03:32:19.852115 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.852050    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 03:32:19.928505 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:19.928468    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:19.968695 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.968639    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 03:27:19 +0000 UTC" deadline="2027-12-27 17:44:09.471668194 +0000 UTC"
Jul 10 03:32:19.968695 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:19.968685    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12854h11m49.502986454s"
Jul 10 03:32:20.021869 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:20.021827    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c6f7f9103477741d51b536b17fca268.slice/crio-27df25d97e4a5ae20fbf49f3bb7952b5be235f8d68782d170b6fb635c417baee WatchSource:0}: Error finding container 27df25d97e4a5ae20fbf49f3bb7952b5be235f8d68782d170b6fb635c417baee: Status 404 returned error can't find the container with id 27df25d97e4a5ae20fbf49f3bb7952b5be235f8d68782d170b6fb635c417baee
Jul 10 03:32:20.022383 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:20.022358    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6cf1273741a1c23d1af7ea6ad5bb863.slice/crio-cce1f4674e3f249bad6bd730a543cb90f30dbfd4d4c7863d45dfa707e5782dd2 WatchSource:0}: Error finding container cce1f4674e3f249bad6bd730a543cb90f30dbfd4d4c7863d45dfa707e5782dd2: Status 404 returned error can't find the container with id cce1f4674e3f249bad6bd730a543cb90f30dbfd4d4c7863d45dfa707e5782dd2
Jul 10 03:32:20.028642 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:20.028532    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:20.029034 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.029020    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 03:32:20.128773 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:20.128737    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:20.142908 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.142893    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:20.144125 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.144108    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:20.144217 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.144136    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:20.144217 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.144151    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:20.229041 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:20.229009    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:20.329531 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:20.329503    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found"
Jul 10 03:32:20.381958 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.381929    2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 03:32:20.414761 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.414731    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:20.428537 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.428521    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 03:32:20.429611 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.429598    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal"
Jul 10 03:32:20.437727 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.437708    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 03:32:20.442826 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.442804    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 03:32:20.893957 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.893924    2561 apiserver.go:52] "Watching apiserver"
Jul 10 03:32:20.900623 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.900598    2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 03:32:20.902795 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.902770    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-qf47j","openshift-image-registry/node-ca-4h4m7","openshift-multus/multus-additional-cni-plugins-5st2k","openshift-multus/network-metrics-daemon-p7l8g","openshift-network-operator/iptables-alerter-w86pn","openshift-ovn-kubernetes/ovnkube-node-knklw","kube-system/global-pull-secret-syncer-lzrdd","kube-system/konnectivity-agent-47lw8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv","openshift-dns/node-resolver-gz9tm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal","openshift-multus/multus-rf6sd","openshift-network-diagnostics/network-check-target-sphtj","kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal"]
Jul 10 03:32:20.905280 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.905254    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:20.905394 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:20.905341    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:20.907397 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.907376    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:20.907499 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:20.907447    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:20.909478 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.909457    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:20.911616 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.911583    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:20.911727 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:20.911701    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:20.911837 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.911816    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 03:32:20.912137 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.912110    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 03:32:20.912257 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.912212    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qnc2d\""
Jul 10 03:32:20.912257 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.912110    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 03:32:20.912363 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.912261    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 03:32:20.912363 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.912343    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 03:32:20.913790 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.913774    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w86pn"
Jul 10 03:32:20.915241 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.915213    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 03:32:20.915992 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.915690    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 03:32:20.915992 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.915747    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 03:32:20.915992 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.915821    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z6qcx\""
Jul 10 03:32:20.916190 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.916054    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 03:32:20.916190 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.916124    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.918330 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.918309    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 03:32:20.918453 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.918421    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 03:32:20.918921 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.918903    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2x47q\""
Jul 10 03:32:20.919009 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.918920    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:20.919065 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.919044    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 03:32:20.919800 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.919783    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 03:32:20.919882 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.919797    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 03:32:20.919882 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.919785    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 03:32:20.921084 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.921011    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 03:32:20.921589 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.921301    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:20.921589 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.921350    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 03:32:20.921589 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.921439    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 03:32:20.921589 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.921352    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8t6g8\""
Jul 10 03:32:20.923375 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.923355    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 03:32:20.923865 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.923821    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 03:32:20.923865 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.923832    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gqct6\""
Jul 10 03:32:20.924003 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.923872    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4h4m7"
Jul 10 03:32:20.925305 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925285    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c23b127a-0ca0-4d4f-9de5-ef99040a7805-os-release\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:20.925394 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925319    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-log-socket\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.925454 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925413    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.925509 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925462    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pf78\" (UniqueName: \"kubernetes.io/projected/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-kube-api-access-7pf78\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.925509 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925494    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j68rf\" (UniqueName: \"kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf\") pod \"network-check-target-sphtj\" (UID: \"13ef3741-844f-4dc4-b748-7a96098484b8\") " pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:20.925612 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925516    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c23b127a-0ca0-4d4f-9de5-ef99040a7805-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:20.925612 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925539    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:20.925612 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925584    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-etc-openvswitch\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.925825 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925625    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-env-overrides\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.925825 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925698    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c23b127a-0ca0-4d4f-9de5-ef99040a7805-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:20.925825 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925729    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-slash\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.925825 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925746    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-run-ovn-kubernetes\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.925825 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925760    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-cni-bin\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.925825 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925780    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-var-lib-openvswitch\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.926096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925823    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xwwj\" (UniqueName: \"kubernetes.io/projected/1cc3d536-f186-41a7-a193-96745a584d7d-kube-api-access-8xwwj\") pod \"iptables-alerter-w86pn\" (UID: \"1cc3d536-f186-41a7-a193-96745a584d7d\") " pod="openshift-network-operator/iptables-alerter-w86pn"
Jul 10 03:32:20.926096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925855    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-systemd-units\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.926096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925874    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-run-netns\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.926096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925901    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ed5febd9-f003-483e-bd13-02cf0eeb73b6-kubelet-config\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:20.926096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925926    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:20.926096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.925948    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-ovnkube-config\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.926096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926052    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-ovn-node-metrics-cert\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.926096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926090    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926103    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c23b127a-0ca0-4d4f-9de5-ef99040a7805-cni-binary-copy\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926149    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs6d2\" (UniqueName: \"kubernetes.io/projected/c23b127a-0ca0-4d4f-9de5-ef99040a7805-kube-api-access-fs6d2\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926173    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78fg4\" (UniqueName: \"kubernetes.io/projected/776389fa-994a-4f3b-bcfb-48c486232e9b-kube-api-access-78fg4\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926210    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cbf46\""
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926215    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c23b127a-0ca0-4d4f-9de5-ef99040a7805-system-cni-dir\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926237    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c23b127a-0ca0-4d4f-9de5-ef99040a7805-cnibin\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926271    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c23b127a-0ca0-4d4f-9de5-ef99040a7805-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926297    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926310    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cc3d536-f186-41a7-a193-96745a584d7d-host-slash\") pod \"iptables-alerter-w86pn\" (UID: \"1cc3d536-f186-41a7-a193-96745a584d7d\") " pod="openshift-network-operator/iptables-alerter-w86pn"
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926354    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-kubelet\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926387    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-run-systemd\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926410    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-run-openvswitch\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.926471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.926432    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-cni-netd\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.928642 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.927954    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-node-log\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.928642 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.928008    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-ovnkube-script-lib\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.928642 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.928043    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cc3d536-f186-41a7-a193-96745a584d7d-iptables-alerter-script\") pod \"iptables-alerter-w86pn\" (UID: \"1cc3d536-f186-41a7-a193-96745a584d7d\") " pod="openshift-network-operator/iptables-alerter-w86pn"
Jul 10 03:32:20.928642 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.928198    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-run-ovn\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:20.928642 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.928207    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gz9tm"
Jul 10 03:32:20.928642 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.928229    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ed5febd9-f003-483e-bd13-02cf0eeb73b6-dbus\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:20.928642 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.928296    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 03:32:20.931490 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.930507    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-47lw8"
Jul 10 03:32:20.931490 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.931292    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 03:32:20.932010 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.931783    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 03:32:20.932010 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.931800    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 03:32:20.932010 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.931891    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ldt9j\""
Jul 10 03:32:20.932812 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.932682    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 03:32:20.932909 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.932885    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 03:32:20.932982 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.932918    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qnp25\""
Jul 10 03:32:20.933955 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.933830    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:20.935748 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.935726    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 03:32:20.935946 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.935916    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v4f6v\""
Jul 10 03:32:20.952214 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.952081    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-k5ghw"
Jul 10 03:32:20.961296 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.961275    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-k5ghw"
Jul 10 03:32:20.992753 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.992682    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal" event={"ID":"d6cf1273741a1c23d1af7ea6ad5bb863","Type":"ContainerStarted","Data":"cce1f4674e3f249bad6bd730a543cb90f30dbfd4d4c7863d45dfa707e5782dd2"}
Jul 10 03:32:20.994137 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:20.994105    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" event={"ID":"9c6f7f9103477741d51b536b17fca268","Type":"ContainerStarted","Data":"27df25d97e4a5ae20fbf49f3bb7952b5be235f8d68782d170b6fb635c417baee"}
Jul 10 03:32:21.015292 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.015262    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 03:32:21.015928 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.015908    2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 03:32:21.028738 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.028709    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-ovnkube-script-lib\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.028864 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.028741    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ed5febd9-f003-483e-bd13-02cf0eeb73b6-dbus\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:21.028864 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.028758    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c23b127a-0ca0-4d4f-9de5-ef99040a7805-os-release\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.028967 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.028875    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-tuned\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.028967 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.028909    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4399c5c6-6b1b-4c30-bed8-d7ae494e282b-serviceca\") pod \"node-ca-4h4m7\" (UID: \"4399c5c6-6b1b-4c30-bed8-d7ae494e282b\") " pod="openshift-image-registry/node-ca-4h4m7"
Jul 10 03:32:21.028967 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.028935    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjq2x\" (UniqueName: \"kubernetes.io/projected/7ae35170-0a86-42c1-9c56-120f2281b128-kube-api-access-xjq2x\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.028967 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.028954    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ed5febd9-f003-483e-bd13-02cf0eeb73b6-dbus\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:21.029143 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.028967    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-log-socket\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.029143 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.028995    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c23b127a-0ca0-4d4f-9de5-ef99040a7805-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.029143 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029071    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-log-socket\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.029143 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029076    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-var-lib-kubelet\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.029143 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029123    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1e21a35d-82a3-4576-8a55-207ebdc1595e-konnectivity-ca\") pod \"konnectivity-agent-47lw8\" (UID: \"1e21a35d-82a3-4576-8a55-207ebdc1595e\") " pod="kube-system/konnectivity-agent-47lw8"
Jul 10 03:32:21.029410 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029152    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-hostroot\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.029410 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029153    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c23b127a-0ca0-4d4f-9de5-ef99040a7805-os-release\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.029410 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029178    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-etc-selinux\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.029410 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029190    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c23b127a-0ca0-4d4f-9de5-ef99040a7805-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.029410 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029212    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:21.029410 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029247    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-env-overrides\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.029410 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029369    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55793f24-1bd7-4242-84bb-72813fc57a1a-tmp\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.029410 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.029378    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:21.029787 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029414    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-ovnkube-script-lib\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.029787 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029471    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-run-k8s-cni-cncf-io\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.029787 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029512    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-var-lib-cni-multus\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.029787 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029553    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-multus-conf-dir\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.029787 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029589    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-registration-dir\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.029787 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029623    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-slash\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.029787 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.029678    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs podName:776389fa-994a-4f3b-bcfb-48c486232e9b nodeName:}" failed. No retries permitted until 2026-07-10 03:32:21.52961619 +0000 UTC m=+3.020612064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs") pod "network-metrics-daemon-p7l8g" (UID: "776389fa-994a-4f3b-bcfb-48c486232e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:21.029787 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029686    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-slash\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.029787 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029722    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-run-ovn-kubernetes\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.029787 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029759    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-host\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.029787 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029776    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-run-ovn-kubernetes\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029786    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs6d2\" (UniqueName: \"kubernetes.io/projected/c23b127a-0ca0-4d4f-9de5-ef99040a7805-kube-api-access-fs6d2\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029821    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-systemd\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029822    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-env-overrides\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029839    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvt86\" (UniqueName: \"kubernetes.io/projected/55793f24-1bd7-4242-84bb-72813fc57a1a-kube-api-access-lvt86\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029862    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-cni-bin\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029885    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-var-lib-openvswitch\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029903    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-systemd-units\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029917    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-run-netns\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029923    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-cni-bin\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029963    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-var-lib-openvswitch\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029988    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-systemd-units\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029994    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-run-netns\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.029998    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cs5q\" (UniqueName: \"kubernetes.io/projected/4399c5c6-6b1b-4c30-bed8-d7ae494e282b-kube-api-access-9cs5q\") pod \"node-ca-4h4m7\" (UID: \"4399c5c6-6b1b-4c30-bed8-d7ae494e282b\") " pod="openshift-image-registry/node-ca-4h4m7"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030036    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-ovn-node-metrics-cert\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030062    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c23b127a-0ca0-4d4f-9de5-ef99040a7805-cni-binary-copy\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030088    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-run\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.030283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030121    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-multus-socket-dir-parent\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030180    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ae35170-0a86-42c1-9c56-120f2281b128-multus-daemon-config\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030221    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-sys-fs\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030253    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c23b127a-0ca0-4d4f-9de5-ef99040a7805-system-cni-dir\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030295    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-sys\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030324    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-var-lib-cni-bin\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030350    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-run-systemd\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030373    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw7l7\" (UniqueName: \"kubernetes.io/projected/0d61a08c-ae16-4639-8e0c-327a1a81d48f-kube-api-access-pw7l7\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030433    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-run-systemd\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030438    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-node-log\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030472    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cc3d536-f186-41a7-a193-96745a584d7d-iptables-alerter-script\") pod \"iptables-alerter-w86pn\" (UID: \"1cc3d536-f186-41a7-a193-96745a584d7d\") " pod="openshift-network-operator/iptables-alerter-w86pn"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030510    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c23b127a-0ca0-4d4f-9de5-ef99040a7805-system-cni-dir\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030517    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c23b127a-0ca0-4d4f-9de5-ef99040a7805-cni-binary-copy\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030536    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030543    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-run-ovn\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030587    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-node-log\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030614    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-modprobe-d\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.031014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030631    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwfwt\" (UniqueName: \"kubernetes.io/projected/7aec8603-2c61-4c91-936d-728b9f3c641f-kube-api-access-kwfwt\") pod \"node-resolver-gz9tm\" (UID: \"7aec8603-2c61-4c91-936d-728b9f3c641f\") " pod="openshift-dns/node-resolver-gz9tm"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030686    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030689    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-run-ovn\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030717    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pf78\" (UniqueName: \"kubernetes.io/projected/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-kube-api-access-7pf78\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030764    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030791    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j68rf\" (UniqueName: \"kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf\") pod \"network-check-target-sphtj\" (UID: \"13ef3741-844f-4dc4-b748-7a96098484b8\") " pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030933    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-kubernetes\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030951    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-system-cni-dir\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.030987    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ae35170-0a86-42c1-9c56-120f2281b128-cni-binary-copy\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031110    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-etc-openvswitch\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031145    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c23b127a-0ca0-4d4f-9de5-ef99040a7805-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031167    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031190    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7aec8603-2c61-4c91-936d-728b9f3c641f-tmp-dir\") pod \"node-resolver-gz9tm\" (UID: \"7aec8603-2c61-4c91-936d-728b9f3c641f\") " pod="openshift-dns/node-resolver-gz9tm"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031215    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-os-release\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031240    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-socket-dir\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031266    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xwwj\" (UniqueName: \"kubernetes.io/projected/1cc3d536-f186-41a7-a193-96745a584d7d-kube-api-access-8xwwj\") pod \"iptables-alerter-w86pn\" (UID: \"1cc3d536-f186-41a7-a193-96745a584d7d\") " pod="openshift-network-operator/iptables-alerter-w86pn"
Jul 10 03:32:21.031679 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031305    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ed5febd9-f003-483e-bd13-02cf0eeb73b6-kubelet-config\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031329    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031351    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-run-multus-certs\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031374    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-device-dir\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031377    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cc3d536-f186-41a7-a193-96745a584d7d-iptables-alerter-script\") pod \"iptables-alerter-w86pn\" (UID: \"1cc3d536-f186-41a7-a193-96745a584d7d\") " pod="openshift-network-operator/iptables-alerter-w86pn"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031402    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-ovnkube-config\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031426    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-sysctl-conf\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031448    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-etc-openvswitch\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031449    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-lib-modules\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031501    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1e21a35d-82a3-4576-8a55-207ebdc1595e-agent-certs\") pod \"konnectivity-agent-47lw8\" (UID: \"1e21a35d-82a3-4576-8a55-207ebdc1595e\") " pod="kube-system/konnectivity-agent-47lw8"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031525    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7aec8603-2c61-4c91-936d-728b9f3c641f-hosts-file\") pod \"node-resolver-gz9tm\" (UID: \"7aec8603-2c61-4c91-936d-728b9f3c641f\") " pod="openshift-dns/node-resolver-gz9tm"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031543    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-cnibin\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031562    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-run-netns\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031620    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-var-lib-kubelet\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031681    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ed5febd9-f003-483e-bd13-02cf0eeb73b6-kubelet-config\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031724    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c23b127a-0ca0-4d4f-9de5-ef99040a7805-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031745    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78fg4\" (UniqueName: \"kubernetes.io/projected/776389fa-994a-4f3b-bcfb-48c486232e9b-kube-api-access-78fg4\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:21.032299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031774    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c23b127a-0ca0-4d4f-9de5-ef99040a7805-cnibin\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.031782    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031890    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c23b127a-0ca0-4d4f-9de5-ef99040a7805-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031923    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-sysconfig\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.031949    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-sysctl-d\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032017    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c23b127a-0ca0-4d4f-9de5-ef99040a7805-cnibin\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032022    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-etc-kubernetes\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.032031    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret podName:ed5febd9-f003-483e-bd13-02cf0eeb73b6 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:21.532015513 +0000 UTC m=+3.023011379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret") pod "global-pull-secret-syncer-lzrdd" (UID: "ed5febd9-f003-483e-bd13-02cf0eeb73b6") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032078    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-ovnkube-config\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032086    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cc3d536-f186-41a7-a193-96745a584d7d-host-slash\") pod \"iptables-alerter-w86pn\" (UID: \"1cc3d536-f186-41a7-a193-96745a584d7d\") " pod="openshift-network-operator/iptables-alerter-w86pn"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032146    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-kubelet\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032203    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cc3d536-f186-41a7-a193-96745a584d7d-host-slash\") pod \"iptables-alerter-w86pn\" (UID: \"1cc3d536-f186-41a7-a193-96745a584d7d\") " pod="openshift-network-operator/iptables-alerter-w86pn"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032210    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-run-openvswitch\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032245    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-kubelet\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032275    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-cni-netd\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032287    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-run-openvswitch\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032317    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4399c5c6-6b1b-4c30-bed8-d7ae494e282b-host\") pod \"node-ca-4h4m7\" (UID: \"4399c5c6-6b1b-4c30-bed8-d7ae494e282b\") " pod="openshift-image-registry/node-ca-4h4m7"
Jul 10 03:32:21.033062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032363    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-multus-cni-dir\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.033912 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.032364    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-host-cni-netd\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.033912 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.033224    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c23b127a-0ca0-4d4f-9de5-ef99040a7805-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.034498 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.034477    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-ovn-node-metrics-cert\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.061786 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.061760    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs6d2\" (UniqueName: \"kubernetes.io/projected/c23b127a-0ca0-4d4f-9de5-ef99040a7805-kube-api-access-fs6d2\") pod \"multus-additional-cni-plugins-5st2k\" (UID: \"c23b127a-0ca0-4d4f-9de5-ef99040a7805\") " pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.063904 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.063879    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:32:21.064022 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.063908    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:32:21.064022 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.063926    2561 projected.go:196] Error preparing data for projected volume kube-api-access-j68rf for pod openshift-network-diagnostics/network-check-target-sphtj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:21.064132 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.064033    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf podName:13ef3741-844f-4dc4-b748-7a96098484b8 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:21.564014477 +0000 UTC m=+3.055010351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-j68rf" (UniqueName: "kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf") pod "network-check-target-sphtj" (UID: "13ef3741-844f-4dc4-b748-7a96098484b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:21.065371 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.065347    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78fg4\" (UniqueName: \"kubernetes.io/projected/776389fa-994a-4f3b-bcfb-48c486232e9b-kube-api-access-78fg4\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:21.068713 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.068016    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xwwj\" (UniqueName: \"kubernetes.io/projected/1cc3d536-f186-41a7-a193-96745a584d7d-kube-api-access-8xwwj\") pod \"iptables-alerter-w86pn\" (UID: \"1cc3d536-f186-41a7-a193-96745a584d7d\") " pod="openshift-network-operator/iptables-alerter-w86pn"
Jul 10 03:32:21.069562 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.069538    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pf78\" (UniqueName: \"kubernetes.io/projected/f6dacccf-f51b-4d6b-876a-90e1bdacaa97-kube-api-access-7pf78\") pod \"ovnkube-node-knklw\" (UID: \"f6dacccf-f51b-4d6b-876a-90e1bdacaa97\") " pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.133128 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133097    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.133322 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133138    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7aec8603-2c61-4c91-936d-728b9f3c641f-tmp-dir\") pod \"node-resolver-gz9tm\" (UID: \"7aec8603-2c61-4c91-936d-728b9f3c641f\") " pod="openshift-dns/node-resolver-gz9tm"
Jul 10 03:32:21.133322 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133160    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-os-release\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.133322 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133244    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.133322 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133261    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-socket-dir\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.133322 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133316    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-run-multus-certs\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133336    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-os-release\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133345    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-device-dir\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133377    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-sysctl-conf\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133392    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-device-dir\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133403    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-lib-modules\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133421    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-socket-dir\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133429    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1e21a35d-82a3-4576-8a55-207ebdc1595e-agent-certs\") pod \"konnectivity-agent-47lw8\" (UID: \"1e21a35d-82a3-4576-8a55-207ebdc1595e\") " pod="kube-system/konnectivity-agent-47lw8"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133435    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-run-multus-certs\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133456    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7aec8603-2c61-4c91-936d-728b9f3c641f-hosts-file\") pod \"node-resolver-gz9tm\" (UID: \"7aec8603-2c61-4c91-936d-728b9f3c641f\") " pod="openshift-dns/node-resolver-gz9tm"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133482    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-cnibin\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133488    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7aec8603-2c61-4c91-936d-728b9f3c641f-tmp-dir\") pod \"node-resolver-gz9tm\" (UID: \"7aec8603-2c61-4c91-936d-728b9f3c641f\") " pod="openshift-dns/node-resolver-gz9tm"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133502    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-run-netns\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133527    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-var-lib-kubelet\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.133564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133554    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-sysconfig\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133575    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-sysctl-conf\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133580    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-sysctl-d\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133578    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7aec8603-2c61-4c91-936d-728b9f3c641f-hosts-file\") pod \"node-resolver-gz9tm\" (UID: \"7aec8603-2c61-4c91-936d-728b9f3c641f\") " pod="openshift-dns/node-resolver-gz9tm"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133615    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-etc-kubernetes\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133619    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-run-netns\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133639    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-sysconfig\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133646    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4399c5c6-6b1b-4c30-bed8-d7ae494e282b-host\") pod \"node-ca-4h4m7\" (UID: \"4399c5c6-6b1b-4c30-bed8-d7ae494e282b\") " pod="openshift-image-registry/node-ca-4h4m7"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133668    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-multus-cni-dir\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133702    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-cnibin\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133715    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-tuned\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133733    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-sysctl-d\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133741    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4399c5c6-6b1b-4c30-bed8-d7ae494e282b-serviceca\") pod \"node-ca-4h4m7\" (UID: \"4399c5c6-6b1b-4c30-bed8-d7ae494e282b\") " pod="openshift-image-registry/node-ca-4h4m7"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133575    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-lib-modules\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133750    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-var-lib-kubelet\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133772    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjq2x\" (UniqueName: \"kubernetes.io/projected/7ae35170-0a86-42c1-9c56-120f2281b128-kube-api-access-xjq2x\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133801    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-var-lib-kubelet\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133826    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1e21a35d-82a3-4576-8a55-207ebdc1595e-konnectivity-ca\") pod \"konnectivity-agent-47lw8\" (UID: \"1e21a35d-82a3-4576-8a55-207ebdc1595e\") " pod="kube-system/konnectivity-agent-47lw8"
Jul 10 03:32:21.134184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133850    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-hostroot\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133872    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-etc-selinux\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133911    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55793f24-1bd7-4242-84bb-72813fc57a1a-tmp\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133951    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-run-k8s-cni-cncf-io\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.133978    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-var-lib-cni-multus\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134005    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-multus-conf-dir\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134032    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-registration-dir\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134060    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-host\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134086    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-systemd\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134109    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvt86\" (UniqueName: \"kubernetes.io/projected/55793f24-1bd7-4242-84bb-72813fc57a1a-kube-api-access-lvt86\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134140    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cs5q\" (UniqueName: \"kubernetes.io/projected/4399c5c6-6b1b-4c30-bed8-d7ae494e282b-kube-api-access-9cs5q\") pod \"node-ca-4h4m7\" (UID: \"4399c5c6-6b1b-4c30-bed8-d7ae494e282b\") " pod="openshift-image-registry/node-ca-4h4m7"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134165    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-run\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134219    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4399c5c6-6b1b-4c30-bed8-d7ae494e282b-serviceca\") pod \"node-ca-4h4m7\" (UID: \"4399c5c6-6b1b-4c30-bed8-d7ae494e282b\") " pod="openshift-image-registry/node-ca-4h4m7"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134223    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-run\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134251    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-multus-socket-dir-parent\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134294    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4399c5c6-6b1b-4c30-bed8-d7ae494e282b-host\") pod \"node-ca-4h4m7\" (UID: \"4399c5c6-6b1b-4c30-bed8-d7ae494e282b\") " pod="openshift-image-registry/node-ca-4h4m7"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134217    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-etc-kubernetes\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134294    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ae35170-0a86-42c1-9c56-120f2281b128-multus-daemon-config\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.135118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134341    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-etc-selinux\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134343    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-sys-fs\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134382    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-sys\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134398    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-multus-socket-dir-parent\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134410    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-var-lib-cni-bin\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134432    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-var-lib-kubelet\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134257    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-hostroot\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134451    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-var-lib-cni-bin\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134458    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw7l7\" (UniqueName: \"kubernetes.io/projected/0d61a08c-ae16-4639-8e0c-327a1a81d48f-kube-api-access-pw7l7\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134493    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-modprobe-d\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134502    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-registration-dir\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134520    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfwt\" (UniqueName: \"kubernetes.io/projected/7aec8603-2c61-4c91-936d-728b9f3c641f-kube-api-access-kwfwt\") pod \"node-resolver-gz9tm\" (UID: \"7aec8603-2c61-4c91-936d-728b9f3c641f\") " pod="openshift-dns/node-resolver-gz9tm"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134548    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1e21a35d-82a3-4576-8a55-207ebdc1595e-konnectivity-ca\") pod \"konnectivity-agent-47lw8\" (UID: \"1e21a35d-82a3-4576-8a55-207ebdc1595e\") " pod="kube-system/konnectivity-agent-47lw8"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134618    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-run-k8s-cni-cncf-io\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134610    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-host-var-lib-cni-multus\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134561    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-systemd\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134687    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-multus-cni-dir\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134751    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-multus-conf-dir\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134505    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-host\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.136861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134385    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0d61a08c-ae16-4639-8e0c-327a1a81d48f-sys-fs\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.136861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134814    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-kubernetes\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.136861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134846    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-system-cni-dir\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134874    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ae35170-0a86-42c1-9c56-120f2281b128-cni-binary-copy\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134889    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ae35170-0a86-42c1-9c56-120f2281b128-multus-daemon-config\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134961    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-sys\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.136861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.134977    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-kubernetes\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.136861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.135015    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-modprobe-d\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.136861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.135044    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ae35170-0a86-42c1-9c56-120f2281b128-system-cni-dir\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.135428    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ae35170-0a86-42c1-9c56-120f2281b128-cni-binary-copy\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.136861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.136647    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/55793f24-1bd7-4242-84bb-72813fc57a1a-etc-tuned\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.136861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.136795    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1e21a35d-82a3-4576-8a55-207ebdc1595e-agent-certs\") pod \"konnectivity-agent-47lw8\" (UID: \"1e21a35d-82a3-4576-8a55-207ebdc1595e\") " pod="kube-system/konnectivity-agent-47lw8"
Jul 10 03:32:21.137282 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.137128    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55793f24-1bd7-4242-84bb-72813fc57a1a-tmp\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.154106 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.153969    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjq2x\" (UniqueName: \"kubernetes.io/projected/7ae35170-0a86-42c1-9c56-120f2281b128-kube-api-access-xjq2x\") pod \"multus-rf6sd\" (UID: \"7ae35170-0a86-42c1-9c56-120f2281b128\") " pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.154239 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.154165    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwfwt\" (UniqueName: \"kubernetes.io/projected/7aec8603-2c61-4c91-936d-728b9f3c641f-kube-api-access-kwfwt\") pod \"node-resolver-gz9tm\" (UID: \"7aec8603-2c61-4c91-936d-728b9f3c641f\") " pod="openshift-dns/node-resolver-gz9tm"
Jul 10 03:32:21.154239 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.154191    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw7l7\" (UniqueName: \"kubernetes.io/projected/0d61a08c-ae16-4639-8e0c-327a1a81d48f-kube-api-access-pw7l7\") pod \"aws-ebs-csi-driver-node-pmbsv\" (UID: \"0d61a08c-ae16-4639-8e0c-327a1a81d48f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.162803 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.162774    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvt86\" (UniqueName: \"kubernetes.io/projected/55793f24-1bd7-4242-84bb-72813fc57a1a-kube-api-access-lvt86\") pod \"tuned-qf47j\" (UID: \"55793f24-1bd7-4242-84bb-72813fc57a1a\") " pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.162935 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.162851    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cs5q\" (UniqueName: \"kubernetes.io/projected/4399c5c6-6b1b-4c30-bed8-d7ae494e282b-kube-api-access-9cs5q\") pod \"node-ca-4h4m7\" (UID: \"4399c5c6-6b1b-4c30-bed8-d7ae494e282b\") " pod="openshift-image-registry/node-ca-4h4m7"
Jul 10 03:32:21.220397 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.220367    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5st2k"
Jul 10 03:32:21.233405 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.233381    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w86pn"
Jul 10 03:32:21.243451 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.243431    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:21.248313 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.248294    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv"
Jul 10 03:32:21.256142 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.256122    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qf47j"
Jul 10 03:32:21.263950 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.263926    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4h4m7"
Jul 10 03:32:21.270877 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.270855    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gz9tm"
Jul 10 03:32:21.278841 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.278822    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-47lw8"
Jul 10 03:32:21.285661 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.285643    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rf6sd"
Jul 10 03:32:21.538678 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.538548    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:21.538678 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.538651    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:21.538919 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.538771    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:21.538919 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.538827    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:21.538919 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.538861    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret podName:ed5febd9-f003-483e-bd13-02cf0eeb73b6 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:22.53884052 +0000 UTC m=+4.029836398 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret") pod "global-pull-secret-syncer-lzrdd" (UID: "ed5febd9-f003-483e-bd13-02cf0eeb73b6") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:21.538919 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.538890    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs podName:776389fa-994a-4f3b-bcfb-48c486232e9b nodeName:}" failed. No retries permitted until 2026-07-10 03:32:22.538872677 +0000 UTC m=+4.029868539 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs") pod "network-metrics-daemon-p7l8g" (UID: "776389fa-994a-4f3b-bcfb-48c486232e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:21.639942 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.639896    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j68rf\" (UniqueName: \"kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf\") pod \"network-check-target-sphtj\" (UID: \"13ef3741-844f-4dc4-b748-7a96098484b8\") " pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:21.640115 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.640077    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:32:21.640115 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.640104    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:32:21.640115 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.640115    2561 projected.go:196] Error preparing data for projected volume kube-api-access-j68rf for pod openshift-network-diagnostics/network-check-target-sphtj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:21.640264 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.640199    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf podName:13ef3741-844f-4dc4-b748-7a96098484b8 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:22.640158224 +0000 UTC m=+4.131154083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-j68rf" (UniqueName: "kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf") pod "network-check-target-sphtj" (UID: "13ef3741-844f-4dc4-b748-7a96098484b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:21.964734 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.964690    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 03:27:21 +0000 UTC" deadline="2028-02-21 15:18:15.889945525 +0000 UTC"
Jul 10 03:32:21.964734 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.964729    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14195h45m53.925220318s"
Jul 10 03:32:21.988085 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:21.988048    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:21.988248 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:21.988179    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:22.007646 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:22.007616    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23b127a_0ca0_4d4f_9de5_ef99040a7805.slice/crio-e4e55ad3b38ce5575b414bd1b14840e28e40be661feebf9aed7cbcc1960db515 WatchSource:0}: Error finding container e4e55ad3b38ce5575b414bd1b14840e28e40be661feebf9aed7cbcc1960db515: Status 404 returned error can't find the container with id e4e55ad3b38ce5575b414bd1b14840e28e40be661feebf9aed7cbcc1960db515
Jul 10 03:32:22.008755 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:22.008726    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d61a08c_ae16_4639_8e0c_327a1a81d48f.slice/crio-fb0ef756d2e5ddad8b87ce2d79a0b63262e060edeb502e8d14e6b5939b2249d5 WatchSource:0}: Error finding container fb0ef756d2e5ddad8b87ce2d79a0b63262e060edeb502e8d14e6b5939b2249d5: Status 404 returned error can't find the container with id fb0ef756d2e5ddad8b87ce2d79a0b63262e060edeb502e8d14e6b5939b2249d5
Jul 10 03:32:22.010147 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:22.010119    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6dacccf_f51b_4d6b_876a_90e1bdacaa97.slice/crio-d6d0c7658788b459fc1e4ee34c0168fadd98d193399d89fe451d62d3c391c185 WatchSource:0}: Error finding container d6d0c7658788b459fc1e4ee34c0168fadd98d193399d89fe451d62d3c391c185: Status 404 returned error can't find the container with id d6d0c7658788b459fc1e4ee34c0168fadd98d193399d89fe451d62d3c391c185
Jul 10 03:32:22.014043 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:22.014018    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc3d536_f186_41a7_a193_96745a584d7d.slice/crio-8b613d02e6d09703f3fe72136fb62f1d3b9f48800652e06ca96da0ace4095fbb WatchSource:0}: Error finding container 8b613d02e6d09703f3fe72136fb62f1d3b9f48800652e06ca96da0ace4095fbb: Status 404 returned error can't find the container with id 8b613d02e6d09703f3fe72136fb62f1d3b9f48800652e06ca96da0ace4095fbb
Jul 10 03:32:22.015392 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:22.015312    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55793f24_1bd7_4242_84bb_72813fc57a1a.slice/crio-4e1a0207bddeb5f419258cfe4698f59e68dd32eb5609496b3691220a71ca6295 WatchSource:0}: Error finding container 4e1a0207bddeb5f419258cfe4698f59e68dd32eb5609496b3691220a71ca6295: Status 404 returned error can't find the container with id 4e1a0207bddeb5f419258cfe4698f59e68dd32eb5609496b3691220a71ca6295
Jul 10 03:32:22.016170 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:22.016134    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e21a35d_82a3_4576_8a55_207ebdc1595e.slice/crio-48882f04014b92bd7ab3f1bf0fbdc0790e93ad9fdac35b0def023048f930672e WatchSource:0}: Error finding container 48882f04014b92bd7ab3f1bf0fbdc0790e93ad9fdac35b0def023048f930672e: Status 404 returned error can't find the container with id 48882f04014b92bd7ab3f1bf0fbdc0790e93ad9fdac35b0def023048f930672e
Jul 10 03:32:22.017837 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:22.017716    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4399c5c6_6b1b_4c30_bed8_d7ae494e282b.slice/crio-2d25fbd2330b5b76d28b0b72c95db433845adefe1d744da28c4d444b138660dd WatchSource:0}: Error finding container 2d25fbd2330b5b76d28b0b72c95db433845adefe1d744da28c4d444b138660dd: Status 404 returned error can't find the container with id 2d25fbd2330b5b76d28b0b72c95db433845adefe1d744da28c4d444b138660dd
Jul 10 03:32:22.019241 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:22.019214    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aec8603_2c61_4c91_936d_728b9f3c641f.slice/crio-373e369d00f55c19740353228662354068349c7fd30dabae74e3bfdc55d5e4c1 WatchSource:0}: Error finding container 373e369d00f55c19740353228662354068349c7fd30dabae74e3bfdc55d5e4c1: Status 404 returned error can't find the container with id 373e369d00f55c19740353228662354068349c7fd30dabae74e3bfdc55d5e4c1
Jul 10 03:32:22.020542 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:22.020521    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ae35170_0a86_42c1_9c56_120f2281b128.slice/crio-23078f11b24a46756cbd1e095bf624b8b128cfd234cf5cb7923d05729ca333f1 WatchSource:0}: Error finding container 23078f11b24a46756cbd1e095bf624b8b128cfd234cf5cb7923d05729ca333f1: Status 404 returned error can't find the container with id 23078f11b24a46756cbd1e095bf624b8b128cfd234cf5cb7923d05729ca333f1
Jul 10 03:32:22.467860 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:22.467650    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 03:32:22.548940 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:22.548548    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:22.548940 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:22.548634    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:22.548940 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:22.548722    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:22.548940 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:22.548788    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:22.548940 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:22.548803    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret podName:ed5febd9-f003-483e-bd13-02cf0eeb73b6 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:24.548781158 +0000 UTC m=+6.039777028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret") pod "global-pull-secret-syncer-lzrdd" (UID: "ed5febd9-f003-483e-bd13-02cf0eeb73b6") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:22.548940 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:22.548846    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs podName:776389fa-994a-4f3b-bcfb-48c486232e9b nodeName:}" failed. No retries permitted until 2026-07-10 03:32:24.548827428 +0000 UTC m=+6.039823284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs") pod "network-metrics-daemon-p7l8g" (UID: "776389fa-994a-4f3b-bcfb-48c486232e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:22.650194 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:22.649512    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j68rf\" (UniqueName: \"kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf\") pod \"network-check-target-sphtj\" (UID: \"13ef3741-844f-4dc4-b748-7a96098484b8\") " pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:22.650194 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:22.649705    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:32:22.650194 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:22.649729    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:32:22.650194 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:22.649743    2561 projected.go:196] Error preparing data for projected volume kube-api-access-j68rf for pod openshift-network-diagnostics/network-check-target-sphtj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:22.650194 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:22.649809    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf podName:13ef3741-844f-4dc4-b748-7a96098484b8 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:24.649789033 +0000 UTC m=+6.140784901 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-j68rf" (UniqueName: "kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf") pod "network-check-target-sphtj" (UID: "13ef3741-844f-4dc4-b748-7a96098484b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:22.965061 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:22.964988    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 03:27:21 +0000 UTC" deadline="2028-04-14 06:18:08.919772698 +0000 UTC"
Jul 10 03:32:22.965061 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:22.965058    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15458h45m45.954719733s"
Jul 10 03:32:22.990316 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:22.989791    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:22.990316 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:22.989940    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:22.990316 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:22.989978    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:22.990316 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:22.990129    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:23.013763 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.013705    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal" event={"ID":"d6cf1273741a1c23d1af7ea6ad5bb863","Type":"ContainerStarted","Data":"a8fcaca51a3bea7911bda9899d5dbea4804d5b28b6fb2b954971b5cda7440f00"}
Jul 10 03:32:23.021781 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.021097    2561 generic.go:356] "Generic (PLEG): container finished" podID="9c6f7f9103477741d51b536b17fca268" containerID="f3f27bc99a8fbca9ee272aecafa7b723dbc654ccf3b62e3f209c028f167a5350" exitCode=0
Jul 10 03:32:23.021781 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.021185    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" event={"ID":"9c6f7f9103477741d51b536b17fca268","Type":"ContainerDied","Data":"f3f27bc99a8fbca9ee272aecafa7b723dbc654ccf3b62e3f209c028f167a5350"}
Jul 10 03:32:23.030126 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.030035    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rf6sd" event={"ID":"7ae35170-0a86-42c1-9c56-120f2281b128","Type":"ContainerStarted","Data":"23078f11b24a46756cbd1e095bf624b8b128cfd234cf5cb7923d05729ca333f1"}
Jul 10 03:32:23.043861 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.043749    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qf47j" event={"ID":"55793f24-1bd7-4242-84bb-72813fc57a1a","Type":"ContainerStarted","Data":"4e1a0207bddeb5f419258cfe4698f59e68dd32eb5609496b3691220a71ca6295"}
Jul 10 03:32:23.046593 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.046476    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knklw" event={"ID":"f6dacccf-f51b-4d6b-876a-90e1bdacaa97","Type":"ContainerStarted","Data":"d6d0c7658788b459fc1e4ee34c0168fadd98d193399d89fe451d62d3c391c185"}
Jul 10 03:32:23.048630 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.048575    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gz9tm" event={"ID":"7aec8603-2c61-4c91-936d-728b9f3c641f","Type":"ContainerStarted","Data":"373e369d00f55c19740353228662354068349c7fd30dabae74e3bfdc55d5e4c1"}
Jul 10 03:32:23.050898 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.049834    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal" podStartSLOduration=3.049815883 podStartE2EDuration="3.049815883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:23.034645201 +0000 UTC m=+4.525641068" watchObservedRunningTime="2026-07-10 03:32:23.049815883 +0000 UTC m=+4.540811751"
Jul 10 03:32:23.053604 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.053527    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4h4m7" event={"ID":"4399c5c6-6b1b-4c30-bed8-d7ae494e282b","Type":"ContainerStarted","Data":"2d25fbd2330b5b76d28b0b72c95db433845adefe1d744da28c4d444b138660dd"}
Jul 10 03:32:23.069177 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.069140    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-47lw8" event={"ID":"1e21a35d-82a3-4576-8a55-207ebdc1595e","Type":"ContainerStarted","Data":"48882f04014b92bd7ab3f1bf0fbdc0790e93ad9fdac35b0def023048f930672e"}
Jul 10 03:32:23.078448 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.078415    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w86pn" event={"ID":"1cc3d536-f186-41a7-a193-96745a584d7d","Type":"ContainerStarted","Data":"8b613d02e6d09703f3fe72136fb62f1d3b9f48800652e06ca96da0ace4095fbb"}
Jul 10 03:32:23.080619 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.080592    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv" event={"ID":"0d61a08c-ae16-4639-8e0c-327a1a81d48f","Type":"ContainerStarted","Data":"fb0ef756d2e5ddad8b87ce2d79a0b63262e060edeb502e8d14e6b5939b2249d5"}
Jul 10 03:32:23.090603 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.090573    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5st2k" event={"ID":"c23b127a-0ca0-4d4f-9de5-ef99040a7805","Type":"ContainerStarted","Data":"e4e55ad3b38ce5575b414bd1b14840e28e40be661feebf9aed7cbcc1960db515"}
Jul 10 03:32:23.988666 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:23.988633    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:23.989355 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:23.988787    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:24.121900 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:24.121853    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" event={"ID":"9c6f7f9103477741d51b536b17fca268","Type":"ContainerStarted","Data":"501ea0a9695b187bcc70be99ff51f03a6f12a9de28099d2b88cf70289dab8739"}
Jul 10 03:32:24.138267 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:24.138209    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" podStartSLOduration=4.138186926 podStartE2EDuration="4.138186926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:24.138137585 +0000 UTC m=+5.629133463" watchObservedRunningTime="2026-07-10 03:32:24.138186926 +0000 UTC m=+5.629182808"
Jul 10 03:32:24.566150 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:24.566102    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:24.566349 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:24.566189    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:24.566422 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:24.566347    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:24.566422 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:24.566405    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret podName:ed5febd9-f003-483e-bd13-02cf0eeb73b6 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:28.566388418 +0000 UTC m=+10.057384273 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret") pod "global-pull-secret-syncer-lzrdd" (UID: "ed5febd9-f003-483e-bd13-02cf0eeb73b6") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:24.566919 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:24.566899    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:24.567017 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:24.566957    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs podName:776389fa-994a-4f3b-bcfb-48c486232e9b nodeName:}" failed. No retries permitted until 2026-07-10 03:32:28.566940179 +0000 UTC m=+10.057936033 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs") pod "network-metrics-daemon-p7l8g" (UID: "776389fa-994a-4f3b-bcfb-48c486232e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:24.667015 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:24.666977    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j68rf\" (UniqueName: \"kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf\") pod \"network-check-target-sphtj\" (UID: \"13ef3741-844f-4dc4-b748-7a96098484b8\") " pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:24.667199 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:24.667186    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:32:24.667259 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:24.667207    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:32:24.667259 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:24.667220    2561 projected.go:196] Error preparing data for projected volume kube-api-access-j68rf for pod openshift-network-diagnostics/network-check-target-sphtj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:24.667354 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:24.667279    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf podName:13ef3741-844f-4dc4-b748-7a96098484b8 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:28.667260916 +0000 UTC m=+10.158256784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-j68rf" (UniqueName: "kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf") pod "network-check-target-sphtj" (UID: "13ef3741-844f-4dc4-b748-7a96098484b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:24.990509 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:24.990319    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:24.990509 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:24.990449    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:24.991194 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:24.990829    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:24.991194 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:24.990925    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:25.987873 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:25.987832    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:25.988054 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:25.987960    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:26.987947 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:26.987917    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:26.988379 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:26.987962    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:26.988379 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:26.988071    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:26.988488 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:26.988370    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:27.987994 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:27.987954    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:27.988435 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:27.988086    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:28.601275 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:28.601234    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:28.601458 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:28.601308    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:28.601520 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:28.601466    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:28.601566 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:28.601526    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs podName:776389fa-994a-4f3b-bcfb-48c486232e9b nodeName:}" failed. No retries permitted until 2026-07-10 03:32:36.601509001 +0000 UTC m=+18.092504854 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs") pod "network-metrics-daemon-p7l8g" (UID: "776389fa-994a-4f3b-bcfb-48c486232e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:28.601778 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:28.601747    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:28.601893 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:28.601818    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret podName:ed5febd9-f003-483e-bd13-02cf0eeb73b6 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:36.601797516 +0000 UTC m=+18.092793383 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret") pod "global-pull-secret-syncer-lzrdd" (UID: "ed5febd9-f003-483e-bd13-02cf0eeb73b6") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:28.701879 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:28.701844    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j68rf\" (UniqueName: \"kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf\") pod \"network-check-target-sphtj\" (UID: \"13ef3741-844f-4dc4-b748-7a96098484b8\") " pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:28.702066 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:28.702043    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:32:28.702147 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:28.702066    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:32:28.702147 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:28.702081    2561 projected.go:196] Error preparing data for projected volume kube-api-access-j68rf for pod openshift-network-diagnostics/network-check-target-sphtj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:28.702243 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:28.702148    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf podName:13ef3741-844f-4dc4-b748-7a96098484b8 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:36.702129623 +0000 UTC m=+18.193125487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-j68rf" (UniqueName: "kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf") pod "network-check-target-sphtj" (UID: "13ef3741-844f-4dc4-b748-7a96098484b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:28.989125 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:28.988573    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:28.989125 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:28.988728    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:28.989125 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:28.988781    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:28.989125 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:28.988904    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:29.988664 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:29.988619    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:29.988829 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:29.988749    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:30.987983 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:30.987951    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:30.987983 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:30.987972    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:30.988483 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:30.988092    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:30.988483 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:30.988239    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:31.988305 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:31.988262    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:31.988790 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:31.988387    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:32.987781 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:32.987747    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:32.987947 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:32.987747    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:32.988009 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:32.987949    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:32.988083 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:32.988049    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:33.988152 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:33.988115    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:33.988576 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:33.988252    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:34.988702 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:34.988662    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:34.988702 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:34.988699    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:34.989270 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:34.988820    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:34.989270 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:34.988981    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:35.987832 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:35.987791    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:35.988054 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:35.987927    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:36.660779 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:36.660737    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:36.661193 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:36.660835    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:36.661193 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:36.660931    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:36.661193 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:36.660978    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:36.661193 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:36.661006    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs podName:776389fa-994a-4f3b-bcfb-48c486232e9b nodeName:}" failed. No retries permitted until 2026-07-10 03:32:52.660983354 +0000 UTC m=+34.151979237 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs") pod "network-metrics-daemon-p7l8g" (UID: "776389fa-994a-4f3b-bcfb-48c486232e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:36.661193 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:36.661028    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret podName:ed5febd9-f003-483e-bd13-02cf0eeb73b6 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:52.661018087 +0000 UTC m=+34.152013945 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret") pod "global-pull-secret-syncer-lzrdd" (UID: "ed5febd9-f003-483e-bd13-02cf0eeb73b6") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:36.762230 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:36.762198    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j68rf\" (UniqueName: \"kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf\") pod \"network-check-target-sphtj\" (UID: \"13ef3741-844f-4dc4-b748-7a96098484b8\") " pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:36.762409 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:36.762383    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:32:36.762472 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:36.762410    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:32:36.762472 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:36.762423    2561 projected.go:196] Error preparing data for projected volume kube-api-access-j68rf for pod openshift-network-diagnostics/network-check-target-sphtj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:36.762547 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:36.762487    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf podName:13ef3741-844f-4dc4-b748-7a96098484b8 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:52.762468431 +0000 UTC m=+34.253464298 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-j68rf" (UniqueName: "kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf") pod "network-check-target-sphtj" (UID: "13ef3741-844f-4dc4-b748-7a96098484b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:36.988389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:36.988299    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:36.988389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:36.988357    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:36.988594 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:36.988493    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:36.988678 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:36.988608    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:37.988389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:37.988362    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:37.988864 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:37.988479    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:38.989353 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:38.989319    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:38.989801 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:38.989427    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:38.989801 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:38.989490    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:38.989801 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:38.989592    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:39.988299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:39.988257    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:39.988479 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:39.988391    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:40.988815 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:40.988486    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:40.989561 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:40.988529    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:40.989561 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:40.988921    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:40.989561 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:40.988990    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:41.158595 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.158569    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gz9tm" event={"ID":"7aec8603-2c61-4c91-936d-728b9f3c641f","Type":"ContainerStarted","Data":"9177a833bffecfb6b2e2c499c39689ed3e82ae96620bf2da3cb59851519d243e"}
Jul 10 03:32:41.159897 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.159870    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4h4m7" event={"ID":"4399c5c6-6b1b-4c30-bed8-d7ae494e282b","Type":"ContainerStarted","Data":"c9f2f7872474e9a5d1f7783db1d16b606c08262ad7fe8f4cbd7a023189238d1b"}
Jul 10 03:32:41.161090 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.161069    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-47lw8" event={"ID":"1e21a35d-82a3-4576-8a55-207ebdc1595e","Type":"ContainerStarted","Data":"9ef344e244efba33fd017f1a545cc816137357f3d702b8964d335b09dd234618"}
Jul 10 03:32:41.162398 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.162378    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv" event={"ID":"0d61a08c-ae16-4639-8e0c-327a1a81d48f","Type":"ContainerStarted","Data":"8fd05d941afee178ad6a0fa5669a4c90704118f1aa13d8e601487117ba7586c7"}
Jul 10 03:32:41.163690 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.163669    2561 generic.go:356] "Generic (PLEG): container finished" podID="c23b127a-0ca0-4d4f-9de5-ef99040a7805" containerID="224bf734b69b2e9f34ce8872eef1be0c973c5c21ec3416ceee7dcca5e6911607" exitCode=0
Jul 10 03:32:41.163762 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.163697    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5st2k" event={"ID":"c23b127a-0ca0-4d4f-9de5-ef99040a7805","Type":"ContainerDied","Data":"224bf734b69b2e9f34ce8872eef1be0c973c5c21ec3416ceee7dcca5e6911607"}
Jul 10 03:32:41.165066 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.165046    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rf6sd" event={"ID":"7ae35170-0a86-42c1-9c56-120f2281b128","Type":"ContainerStarted","Data":"3bb8feecf2111a14aee8d6a48e434edf831183c2c495f00bb09b2484deb73bb6"}
Jul 10 03:32:41.166498 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.166473    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qf47j" event={"ID":"55793f24-1bd7-4242-84bb-72813fc57a1a","Type":"ContainerStarted","Data":"eb52ad84272c180cc563bf8987ff5c2b87658e548460c20d0162ab8f8b0615d0"}
Jul 10 03:32:41.169051 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.169034    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knklw" event={"ID":"f6dacccf-f51b-4d6b-876a-90e1bdacaa97","Type":"ContainerStarted","Data":"fc41b71c7aaab44ac957ab16c3b19f1f43abcf5f70622915666186b7c91e2ce3"}
Jul 10 03:32:41.169126 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.169085    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knklw" event={"ID":"f6dacccf-f51b-4d6b-876a-90e1bdacaa97","Type":"ContainerStarted","Data":"426a3b2e21f0b7431d3d0b8190d6a74133a87f4bed05d7eadde934342052111d"}
Jul 10 03:32:41.169164 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.169132    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knklw" event={"ID":"f6dacccf-f51b-4d6b-876a-90e1bdacaa97","Type":"ContainerStarted","Data":"0ab2172e108ae1c80ad98301adb767fb2dde5dae9a1d40e1d6d009d142cb3da6"}
Jul 10 03:32:41.169164 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.169161    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knklw" event={"ID":"f6dacccf-f51b-4d6b-876a-90e1bdacaa97","Type":"ContainerStarted","Data":"2270313db41760b182a0c3117478bd383da3e355b458d3f3395eb87b76afe69c"}
Jul 10 03:32:41.172760 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.172723    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gz9tm" podStartSLOduration=11.800846992 podStartE2EDuration="22.172711374s" totalImagesPullingTime="10.371864382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:41.171949914 +0000 UTC m=+22.662945787" watchObservedRunningTime="2026-07-10 03:32:41.172711374 +0000 UTC m=+22.663707247"
Jul 10 03:32:41.184707 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.184672    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4h4m7" podStartSLOduration=3.530247688 podStartE2EDuration="22.184640528s" totalImagesPullingTime="18.65439284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:41.18435741 +0000 UTC m=+22.675353282" watchObservedRunningTime="2026-07-10 03:32:41.184640528 +0000 UTC m=+22.675636401"
Jul 10 03:32:41.198244 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.198209    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-47lw8" podStartSLOduration=11.822717101 podStartE2EDuration="22.198199427s" totalImagesPullingTime="10.375482326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:41.197867743 +0000 UTC m=+22.688863639" watchObservedRunningTime="2026-07-10 03:32:41.198199427 +0000 UTC m=+22.689195300"
Jul 10 03:32:41.215400 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.215354    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rf6sd" podStartSLOduration=3.503308167 podStartE2EDuration="22.215336181s" totalImagesPullingTime="18.712028014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:41.214929246 +0000 UTC m=+22.705925120" watchObservedRunningTime="2026-07-10 03:32:41.215336181 +0000 UTC m=+22.706332055"
Jul 10 03:32:41.255829 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.255783    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qf47j" podStartSLOduration=3.595176931 podStartE2EDuration="22.255767692s" totalImagesPullingTime="18.660590761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:41.255395401 +0000 UTC m=+22.746391284" watchObservedRunningTime="2026-07-10 03:32:41.255767692 +0000 UTC m=+22.746763564"
Jul 10 03:32:41.988525 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:41.988450    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:41.988784 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:41.988552    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:42.173529 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:42.173499    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knklw" event={"ID":"f6dacccf-f51b-4d6b-876a-90e1bdacaa97","Type":"ContainerStarted","Data":"2dd19c04c3fcba13e5bda3f0045baf03084c19e2955b9f371aafb0a95ba42096"}
Jul 10 03:32:42.173875 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:42.173573    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knklw" event={"ID":"f6dacccf-f51b-4d6b-876a-90e1bdacaa97","Type":"ContainerStarted","Data":"3634955da6454a25883b89fd41d47c7f0bdac41f9582f11a0a9c1fdfc3e7eed4"}
Jul 10 03:32:42.174848 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:42.174826    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w86pn" event={"ID":"1cc3d536-f186-41a7-a193-96745a584d7d","Type":"ContainerStarted","Data":"50934a927cbb2bc292ca9891f817379e2686b43f2b0a27b10a44a89f3290391d"}
Jul 10 03:32:42.187766 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:42.187724    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w86pn" podStartSLOduration=4.5764748189999995 podStartE2EDuration="23.187710902s" totalImagesPullingTime="18.611236083s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:42.187315002 +0000 UTC m=+23.678310876" watchObservedRunningTime="2026-07-10 03:32:42.187710902 +0000 UTC m=+23.678706774"
Jul 10 03:32:42.251741 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:42.251683    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-47lw8"
Jul 10 03:32:42.252297 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:42.252276    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-47lw8"
Jul 10 03:32:42.344680 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:42.344641    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 03:32:42.988755 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:42.988723    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:42.988755 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:42.988737    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:42.989239 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:42.988853    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:42.989239 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:42.988979    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:42.997737 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:42.997616    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T03:32:42.344675005Z","UUID":"5a52d6a4-8e80-46dc-9e28-c7b96883484f","Handler":null,"Name":"","Endpoint":""}
Jul 10 03:32:42.999168 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:42.999146    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 03:32:42.999297 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:42.999187    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 03:32:43.179835 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:43.179798    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv" event={"ID":"0d61a08c-ae16-4639-8e0c-327a1a81d48f","Type":"ContainerStarted","Data":"5b1cd19b22cb52ad9eccf830c5bc4d7eba8f0ac179b895a45b1ac50b2fd83241"}
Jul 10 03:32:43.988015 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:43.987832    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:43.988314 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:43.988116    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:44.183495 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:44.183461    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv" event={"ID":"0d61a08c-ae16-4639-8e0c-327a1a81d48f","Type":"ContainerStarted","Data":"3edfd871d2504a910528aceeb164cf76cf75856f9c842fa139f973a1540b116d"}
Jul 10 03:32:44.186513 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:44.186485    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knklw" event={"ID":"f6dacccf-f51b-4d6b-876a-90e1bdacaa97","Type":"ContainerStarted","Data":"9a56eb9710f53b9c93a2b127a47baeb8a6bc67e388e69d0c860a168ff31e5907"}
Jul 10 03:32:44.186625 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:44.186529    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 10 03:32:44.200460 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:44.200413    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmbsv" podStartSLOduration=4.209740075 podStartE2EDuration="25.200395564s" totalImagesPullingTime="20.990655489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:44.199904318 +0000 UTC m=+25.690900192" watchObservedRunningTime="2026-07-10 03:32:44.200395564 +0000 UTC m=+25.691391438"
Jul 10 03:32:44.987749 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:44.987716    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:44.987972 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:44.987861    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:44.988071 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:44.987716    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:44.988190 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:44.988163    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:45.987820 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:45.987786    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:45.988324 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:45.987901    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:46.988284 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:46.988258    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:46.988941 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:46.988386    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:46.988941 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:46.988436    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:46.988941 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:46.988565    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:47.198138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:47.197892    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knklw" event={"ID":"f6dacccf-f51b-4d6b-876a-90e1bdacaa97","Type":"ContainerStarted","Data":"0ad9f21f6e63b0abdaae281f6e789a005683cf3a9a9de98150ae9671192541a9"}
Jul 10 03:32:47.198834 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:47.198205    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:47.198834 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:47.198270    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:47.217344 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:47.217247    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:47.223582 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:47.223519    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-knklw" podStartSLOduration=9.487868538 podStartE2EDuration="28.223505964s" totalImagesPullingTime="18.735637426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:47.222593688 +0000 UTC m=+28.713589586" watchObservedRunningTime="2026-07-10 03:32:47.223505964 +0000 UTC m=+28.714501837"
Jul 10 03:32:47.988095 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:47.988062    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:47.988328 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:47.988168    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:48.203134 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:48.203096    2561 generic.go:356] "Generic (PLEG): container finished" podID="c23b127a-0ca0-4d4f-9de5-ef99040a7805" containerID="f671db3fb1cc5a4621be80749a4b9a0a39e87b7155470f93b0340ca1101465f7" exitCode=0
Jul 10 03:32:48.203516 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:48.203489    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5st2k" event={"ID":"c23b127a-0ca0-4d4f-9de5-ef99040a7805","Type":"ContainerDied","Data":"f671db3fb1cc5a4621be80749a4b9a0a39e87b7155470f93b0340ca1101465f7"}
Jul 10 03:32:48.204035 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:48.204017    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:48.225487 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:48.225457    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:32:48.850437 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:48.850401    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p7l8g"]
Jul 10 03:32:48.850633 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:48.850581    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:48.850748 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:48.850722    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:48.854687 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:48.854634    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lzrdd"]
Jul 10 03:32:48.854857 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:48.854785    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:48.854973 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:48.854950    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:48.855422 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:48.855372    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sphtj"]
Jul 10 03:32:48.855528 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:48.855474    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:48.855589 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:48.855561    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:50.209390 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:50.209137    2561 generic.go:356] "Generic (PLEG): container finished" podID="c23b127a-0ca0-4d4f-9de5-ef99040a7805" containerID="98e90348a275ffa23340cd2e56103e01a55c48c1f3b8fea777b62307b78b526d" exitCode=0
Jul 10 03:32:50.209887 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:50.209178    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5st2k" event={"ID":"c23b127a-0ca0-4d4f-9de5-ef99040a7805","Type":"ContainerDied","Data":"98e90348a275ffa23340cd2e56103e01a55c48c1f3b8fea777b62307b78b526d"}
Jul 10 03:32:50.412221 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:50.412177    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-47lw8"
Jul 10 03:32:50.412392 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:50.412350    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 10 03:32:50.412997 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:50.412976    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-47lw8"
Jul 10 03:32:50.988037 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:50.988006    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:50.988216 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:50.988006    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:50.988216 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:50.988128    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:50.988334 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:50.988209    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:50.988334 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:50.988006    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:50.988334 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:50.988309    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:52.217712 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:52.217677    2561 generic.go:356] "Generic (PLEG): container finished" podID="c23b127a-0ca0-4d4f-9de5-ef99040a7805" containerID="bfd3150d8bd7d2656ce0d85d30a8e8b748939bebbae9ea5ff2cd5be059d3a796" exitCode=0
Jul 10 03:32:52.218360 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:52.217729    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5st2k" event={"ID":"c23b127a-0ca0-4d4f-9de5-ef99040a7805","Type":"ContainerDied","Data":"bfd3150d8bd7d2656ce0d85d30a8e8b748939bebbae9ea5ff2cd5be059d3a796"}
Jul 10 03:32:52.677481 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:52.677450    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:52.677636 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:52.677503    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:52.677636 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:52.677585    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:52.677636 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:52.677590    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:52.677636 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:52.677637    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret podName:ed5febd9-f003-483e-bd13-02cf0eeb73b6 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:24.677624432 +0000 UTC m=+66.168620283 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret") pod "global-pull-secret-syncer-lzrdd" (UID: "ed5febd9-f003-483e-bd13-02cf0eeb73b6") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:52.677801 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:52.677674    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs podName:776389fa-994a-4f3b-bcfb-48c486232e9b nodeName:}" failed. No retries permitted until 2026-07-10 03:33:24.677644229 +0000 UTC m=+66.168640080 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs") pod "network-metrics-daemon-p7l8g" (UID: "776389fa-994a-4f3b-bcfb-48c486232e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:52.778620 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:52.778582    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j68rf\" (UniqueName: \"kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf\") pod \"network-check-target-sphtj\" (UID: \"13ef3741-844f-4dc4-b748-7a96098484b8\") " pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:52.778838 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:52.778758    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:32:52.778838 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:52.778776    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:32:52.778838 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:52.778786    2561 projected.go:196] Error preparing data for projected volume kube-api-access-j68rf for pod openshift-network-diagnostics/network-check-target-sphtj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:52.778838 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:52.778835    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf podName:13ef3741-844f-4dc4-b748-7a96098484b8 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:24.778822242 +0000 UTC m=+66.269818093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-j68rf" (UniqueName: "kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf") pod "network-check-target-sphtj" (UID: "13ef3741-844f-4dc4-b748-7a96098484b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:52.988356 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:52.988216    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:52.988356 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:52.988230    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:52.988575 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:52.988360    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:32:52.988575 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:52.988415    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:52.988575 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:52.988524    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lzrdd" podUID="ed5febd9-f003-483e-bd13-02cf0eeb73b6"
Jul 10 03:32:52.988746 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:52.988604    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sphtj" podUID="13ef3741-844f-4dc4-b748-7a96098484b8"
Jul 10 03:32:53.300278 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.300212    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeReady"
Jul 10 03:32:53.300705 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.300330    2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 03:32:53.334637 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.334595    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"]
Jul 10 03:32:53.339202 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.339179    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f84d8746d-csqp6"]
Jul 10 03:32:53.339318 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.339302    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:32:53.341951 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.341927    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 10 03:32:53.342097 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.341927    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 10 03:32:53.342097 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.342013    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2bqsd\""
Jul 10 03:32:53.343326 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.343305    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.344425 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.344405    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"]
Jul 10 03:32:53.346229 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.346183    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 10 03:32:53.347069 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.346735    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tqxc2\""
Jul 10 03:32:53.347069 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.346931    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 10 03:32:53.348481 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.347942    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 10 03:32:53.348583 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.348564    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f84d8746d-csqp6"]
Jul 10 03:32:53.352149 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.351372    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m9dtr"]
Jul 10 03:32:53.352244 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.352204    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 10 03:32:53.355314 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.355296    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4rqhk"]
Jul 10 03:32:53.355487 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.355430    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m9dtr"
Jul 10 03:32:53.357496 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.357476    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 03:32:53.357602 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.357553    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 03:32:53.357752 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.357733    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7txf6\""
Jul 10 03:32:53.357828 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.357794    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 03:32:53.359104 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.359083    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:53.361195 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.361177    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dk2pk\""
Jul 10 03:32:53.361408 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.361388    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 03:32:53.361694 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.361623    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m9dtr"]
Jul 10 03:32:53.361694 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.361644    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 03:32:53.363783 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.363735    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4rqhk"]
Jul 10 03:32:53.484814 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.484779    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32888fa2-39e4-4de1-9a19-dee518a05139-registry-certificates\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.484814 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.484813    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32888fa2-39e4-4de1-9a19-dee518a05139-installation-pull-secrets\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.485081 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.484832    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0843417a-dbe0-45e1-96c6-4510eb4d74ef-config-volume\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:53.485081 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.484903    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ed31fb70-7e2b-4d84-946b-b00431c7e731-nginx-conf\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:32:53.485081 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.484937    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32888fa2-39e4-4de1-9a19-dee518a05139-ca-trust-extracted\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.485081 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.484956    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbljc\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-kube-api-access-kbljc\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.485081 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.484981    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32888fa2-39e4-4de1-9a19-dee518a05139-trusted-ca\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.485081 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.484998    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0843417a-dbe0-45e1-96c6-4510eb4d74ef-tmp-dir\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:53.485081 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.485016    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:53.485081 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.485033    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.485081 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.485083    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-bound-sa-token\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.485363 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.485109    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32888fa2-39e4-4de1-9a19-dee518a05139-image-registry-private-configuration\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.485363 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.485127    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtcsx\" (UniqueName: \"kubernetes.io/projected/0843417a-dbe0-45e1-96c6-4510eb4d74ef-kube-api-access-dtcsx\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:53.485363 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.485143    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:32:53.485363 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.485157    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39488c5e-12b6-4a76-9383-28aa9c74d14c-cert\") pod \"ingress-canary-m9dtr\" (UID: \"39488c5e-12b6-4a76-9383-28aa9c74d14c\") " pod="openshift-ingress-canary/ingress-canary-m9dtr"
Jul 10 03:32:53.485363 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.485171    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nngv\" (UniqueName: \"kubernetes.io/projected/39488c5e-12b6-4a76-9383-28aa9c74d14c-kube-api-access-9nngv\") pod \"ingress-canary-m9dtr\" (UID: \"39488c5e-12b6-4a76-9383-28aa9c74d14c\") " pod="openshift-ingress-canary/ingress-canary-m9dtr"
Jul 10 03:32:53.586136 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586086    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:53.586323 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586146    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.586323 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586196    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-bound-sa-token\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.586323 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586238    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32888fa2-39e4-4de1-9a19-dee518a05139-image-registry-private-configuration\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.586323 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586267    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtcsx\" (UniqueName: \"kubernetes.io/projected/0843417a-dbe0-45e1-96c6-4510eb4d74ef-kube-api-access-dtcsx\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:53.586323 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:53.586274    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:32:53.586323 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586297    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:32:53.586323 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586325    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39488c5e-12b6-4a76-9383-28aa9c74d14c-cert\") pod \"ingress-canary-m9dtr\" (UID: \"39488c5e-12b6-4a76-9383-28aa9c74d14c\") " pod="openshift-ingress-canary/ingress-canary-m9dtr"
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:53.586329    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:53.586348    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f84d8746d-csqp6: secret "image-registry-tls" not found
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:53.586361    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls podName:0843417a-dbe0-45e1-96c6-4510eb4d74ef nodeName:}" failed. No retries permitted until 2026-07-10 03:32:54.086338126 +0000 UTC m=+35.577333991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls") pod "dns-default-4rqhk" (UID: "0843417a-dbe0-45e1-96c6-4510eb4d74ef") : secret "dns-default-metrics-tls" not found
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586386    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nngv\" (UniqueName: \"kubernetes.io/projected/39488c5e-12b6-4a76-9383-28aa9c74d14c-kube-api-access-9nngv\") pod \"ingress-canary-m9dtr\" (UID: \"39488c5e-12b6-4a76-9383-28aa9c74d14c\") " pod="openshift-ingress-canary/ingress-canary-m9dtr"
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:53.586410    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls podName:32888fa2-39e4-4de1-9a19-dee518a05139 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:54.086392258 +0000 UTC m=+35.577388114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls") pod "image-registry-7f84d8746d-csqp6" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139") : secret "image-registry-tls" not found
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586463    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32888fa2-39e4-4de1-9a19-dee518a05139-registry-certificates\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586497    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32888fa2-39e4-4de1-9a19-dee518a05139-installation-pull-secrets\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586521    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0843417a-dbe0-45e1-96c6-4510eb4d74ef-config-volume\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586567    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ed31fb70-7e2b-4d84-946b-b00431c7e731-nginx-conf\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586606    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32888fa2-39e4-4de1-9a19-dee518a05139-ca-trust-extracted\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586635    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbljc\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-kube-api-access-kbljc\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586683    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32888fa2-39e4-4de1-9a19-dee518a05139-trusted-ca\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.586699 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.586706    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0843417a-dbe0-45e1-96c6-4510eb4d74ef-tmp-dir\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:53.587320 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.587022    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0843417a-dbe0-45e1-96c6-4510eb4d74ef-tmp-dir\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:53.587320 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:53.587047    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 03:32:53.587320 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:53.587095    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert podName:ed31fb70-7e2b-4d84-946b-b00431c7e731 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:54.087080923 +0000 UTC m=+35.578076776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9gjc5" (UID: "ed31fb70-7e2b-4d84-946b-b00431c7e731") : secret "networking-console-plugin-cert" not found
Jul 10 03:32:53.587468 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.587359    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32888fa2-39e4-4de1-9a19-dee518a05139-registry-certificates\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.587664 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.587537    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32888fa2-39e4-4de1-9a19-dee518a05139-ca-trust-extracted\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.587821 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.587772    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ed31fb70-7e2b-4d84-946b-b00431c7e731-nginx-conf\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:32:53.587978 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.587954    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0843417a-dbe0-45e1-96c6-4510eb4d74ef-config-volume\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:53.588063 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.587964    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32888fa2-39e4-4de1-9a19-dee518a05139-trusted-ca\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.591517 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.591491    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32888fa2-39e4-4de1-9a19-dee518a05139-image-registry-private-configuration\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.591607 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.591564    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32888fa2-39e4-4de1-9a19-dee518a05139-installation-pull-secrets\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.591607 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.591590    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39488c5e-12b6-4a76-9383-28aa9c74d14c-cert\") pod \"ingress-canary-m9dtr\" (UID: \"39488c5e-12b6-4a76-9383-28aa9c74d14c\") " pod="openshift-ingress-canary/ingress-canary-m9dtr"
Jul 10 03:32:53.595467 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.595449    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nngv\" (UniqueName: \"kubernetes.io/projected/39488c5e-12b6-4a76-9383-28aa9c74d14c-kube-api-access-9nngv\") pod \"ingress-canary-m9dtr\" (UID: \"39488c5e-12b6-4a76-9383-28aa9c74d14c\") " pod="openshift-ingress-canary/ingress-canary-m9dtr"
Jul 10 03:32:53.595676 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.595471    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbljc\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-kube-api-access-kbljc\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.595828 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.595807    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-bound-sa-token\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:53.595985 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.595964    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtcsx\" (UniqueName: \"kubernetes.io/projected/0843417a-dbe0-45e1-96c6-4510eb4d74ef-kube-api-access-dtcsx\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:53.671081 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.671052    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m9dtr"
Jul 10 03:32:53.813710 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:53.813454    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m9dtr"]
Jul 10 03:32:53.818624 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:32:53.818590    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39488c5e_12b6_4a76_9383_28aa9c74d14c.slice/crio-c5e8d6ecc20368898c6aaabaf8aa45ad8e3029d0d7191ba9302b59d3c51e2606 WatchSource:0}: Error finding container c5e8d6ecc20368898c6aaabaf8aa45ad8e3029d0d7191ba9302b59d3c51e2606: Status 404 returned error can't find the container with id c5e8d6ecc20368898c6aaabaf8aa45ad8e3029d0d7191ba9302b59d3c51e2606
Jul 10 03:32:54.091338 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.091302    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:32:54.091516 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.091425    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:54.091516 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.091456    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:54.094550 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:54.091806    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 03:32:54.094550 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:54.091920    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert podName:ed31fb70-7e2b-4d84-946b-b00431c7e731 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:55.091887834 +0000 UTC m=+36.582883701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9gjc5" (UID: "ed31fb70-7e2b-4d84-946b-b00431c7e731") : secret "networking-console-plugin-cert" not found
Jul 10 03:32:54.094550 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:54.091943    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 03:32:54.094550 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:54.091962    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f84d8746d-csqp6: secret "image-registry-tls" not found
Jul 10 03:32:54.094550 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:54.092022    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:32:54.094550 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:54.092025    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls podName:32888fa2-39e4-4de1-9a19-dee518a05139 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:55.092007913 +0000 UTC m=+36.583003787 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls") pod "image-registry-7f84d8746d-csqp6" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139") : secret "image-registry-tls" not found
Jul 10 03:32:54.094550 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:54.092092    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls podName:0843417a-dbe0-45e1-96c6-4510eb4d74ef nodeName:}" failed. No retries permitted until 2026-07-10 03:32:55.092077691 +0000 UTC m=+36.583073553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls") pod "dns-default-4rqhk" (UID: "0843417a-dbe0-45e1-96c6-4510eb4d74ef") : secret "dns-default-metrics-tls" not found
Jul 10 03:32:54.224227 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.224191    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m9dtr" event={"ID":"39488c5e-12b6-4a76-9383-28aa9c74d14c","Type":"ContainerStarted","Data":"c5e8d6ecc20368898c6aaabaf8aa45ad8e3029d0d7191ba9302b59d3c51e2606"}
Jul 10 03:32:54.988206 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.988167    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:32:54.988206 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.988195    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:32:54.988734 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.988185    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:32:54.990779 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.990758    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 03:32:54.991544 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.991523    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 03:32:54.991678 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.991558    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sg5lf\""
Jul 10 03:32:54.991678 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.991572    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 03:32:54.991678 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.991588    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c4cxw\""
Jul 10 03:32:54.991934 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:54.991915    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 03:32:55.102017 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:55.101986    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:32:55.102203 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:55.102097    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:55.102203 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:55.102129    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:55.102318 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:55.102283    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 03:32:55.102318 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:55.102301    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f84d8746d-csqp6: secret "image-registry-tls" not found
Jul 10 03:32:55.102414 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:55.102372    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls podName:32888fa2-39e4-4de1-9a19-dee518a05139 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:57.10235156 +0000 UTC m=+38.593347429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls") pod "image-registry-7f84d8746d-csqp6" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139") : secret "image-registry-tls" not found
Jul 10 03:32:55.102655 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:55.102628    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:32:55.102790 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:55.102688    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 03:32:55.102790 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:55.102710    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls podName:0843417a-dbe0-45e1-96c6-4510eb4d74ef nodeName:}" failed. No retries permitted until 2026-07-10 03:32:57.102693573 +0000 UTC m=+38.593689440 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls") pod "dns-default-4rqhk" (UID: "0843417a-dbe0-45e1-96c6-4510eb4d74ef") : secret "dns-default-metrics-tls" not found
Jul 10 03:32:55.102790 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:55.102738    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert podName:ed31fb70-7e2b-4d84-946b-b00431c7e731 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:57.102724502 +0000 UTC m=+38.593720356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9gjc5" (UID: "ed31fb70-7e2b-4d84-946b-b00431c7e731") : secret "networking-console-plugin-cert" not found
Jul 10 03:32:57.117065 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:57.117025    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:32:57.117578 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:57.117077    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:32:57.117578 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:57.117136    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:32:57.117578 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:57.117185    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:32:57.117578 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:57.117200    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 03:32:57.117578 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:57.117218    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f84d8746d-csqp6: secret "image-registry-tls" not found
Jul 10 03:32:57.117578 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:57.117254    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls podName:0843417a-dbe0-45e1-96c6-4510eb4d74ef nodeName:}" failed. No retries permitted until 2026-07-10 03:33:01.117232922 +0000 UTC m=+42.608228787 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls") pod "dns-default-4rqhk" (UID: "0843417a-dbe0-45e1-96c6-4510eb4d74ef") : secret "dns-default-metrics-tls" not found
Jul 10 03:32:57.117578 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:57.117275    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls podName:32888fa2-39e4-4de1-9a19-dee518a05139 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:01.117265863 +0000 UTC m=+42.608261720 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls") pod "image-registry-7f84d8746d-csqp6" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139") : secret "image-registry-tls" not found
Jul 10 03:32:57.117578 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:57.117283    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 03:32:57.117578 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:32:57.117332    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert podName:ed31fb70-7e2b-4d84-946b-b00431c7e731 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:01.117317217 +0000 UTC m=+42.608313070 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9gjc5" (UID: "ed31fb70-7e2b-4d84-946b-b00431c7e731") : secret "networking-console-plugin-cert" not found
Jul 10 03:32:57.916572 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:57.916520    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:32:58.008224 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:58.008191    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:32:59.237671 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:59.237624    2561 generic.go:356] "Generic (PLEG): container finished" podID="c23b127a-0ca0-4d4f-9de5-ef99040a7805" containerID="0602d55fc057b272551c96897011e10f0d16deabc156e04357e93d1385ded978" exitCode=0
Jul 10 03:32:59.238145 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:59.237708    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5st2k" event={"ID":"c23b127a-0ca0-4d4f-9de5-ef99040a7805","Type":"ContainerDied","Data":"0602d55fc057b272551c96897011e10f0d16deabc156e04357e93d1385ded978"}
Jul 10 03:32:59.239258 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:59.239234    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m9dtr" event={"ID":"39488c5e-12b6-4a76-9383-28aa9c74d14c","Type":"ContainerStarted","Data":"7f18ab2e1e1eb47f0fb5fc156af8fe5391763ad9c4f78c2014b785cb607e3df8"}
Jul 10 03:32:59.272557 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:32:59.272508    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m9dtr" podStartSLOduration=1.872098275 podStartE2EDuration="6.272487606s" totalImagesPullingTime="4.400389331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:59.271957019 +0000 UTC m=+40.762952894" watchObservedRunningTime="2026-07-10 03:32:59.272487606 +0000 UTC m=+40.763483472"
Jul 10 03:33:00.244405 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:00.244374    2561 generic.go:356] "Generic (PLEG): container finished" podID="c23b127a-0ca0-4d4f-9de5-ef99040a7805" containerID="790cd8b36c31d03c8cbba0727453bd2592678437f0e6962ac338147f117ea32c" exitCode=0
Jul 10 03:33:00.244769 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:00.244459    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5st2k" event={"ID":"c23b127a-0ca0-4d4f-9de5-ef99040a7805","Type":"ContainerDied","Data":"790cd8b36c31d03c8cbba0727453bd2592678437f0e6962ac338147f117ea32c"}
Jul 10 03:33:00.316577 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:00.316533    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:00.369909 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:00.369870    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:00.452694 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:00.452660    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:01.149927 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:01.149878    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:33:01.149927 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:01.149922    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:33:01.150161 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:01.150032    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:33:01.150161 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:01.150057    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:33:01.150161 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:01.150145    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls podName:0843417a-dbe0-45e1-96c6-4510eb4d74ef nodeName:}" failed. No retries permitted until 2026-07-10 03:33:09.150121143 +0000 UTC m=+50.641116996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls") pod "dns-default-4rqhk" (UID: "0843417a-dbe0-45e1-96c6-4510eb4d74ef") : secret "dns-default-metrics-tls" not found
Jul 10 03:33:01.150161 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:01.150145    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 03:33:01.150324 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:01.150162    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 03:33:01.150324 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:01.150178    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f84d8746d-csqp6: secret "image-registry-tls" not found
Jul 10 03:33:01.150324 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:01.150187    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert podName:ed31fb70-7e2b-4d84-946b-b00431c7e731 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:09.15017703 +0000 UTC m=+50.641172882 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9gjc5" (UID: "ed31fb70-7e2b-4d84-946b-b00431c7e731") : secret "networking-console-plugin-cert" not found
Jul 10 03:33:01.150324 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:01.150219    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls podName:32888fa2-39e4-4de1-9a19-dee518a05139 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:09.150205497 +0000 UTC m=+50.641201351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls") pod "image-registry-7f84d8746d-csqp6" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139") : secret "image-registry-tls" not found
Jul 10 03:33:01.249721 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:01.249686    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5st2k" event={"ID":"c23b127a-0ca0-4d4f-9de5-ef99040a7805","Type":"ContainerStarted","Data":"80429500a126de6e1a408d1b1bc3408aae8151befcf13d668ca0efb9cd5386eb"}
Jul 10 03:33:01.272186 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:01.272139    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5st2k" podStartSLOduration=9.193153201 podStartE2EDuration="42.272120561s" totalImagesPullingTime="33.07896736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:01.270827922 +0000 UTC m=+42.761823795" watchObservedRunningTime="2026-07-10 03:33:01.272120561 +0000 UTC m=+42.763116433"
Jul 10 03:33:01.400288 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:01.400203    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:01.500376 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:01.500333    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:01.555937 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:01.555906    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:01.685333 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:01.685257    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:01.783453 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:01.783423    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:08.422844 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:08.422812    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:08.523482 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:08.523449    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:09.098456 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:09.098422    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:09.200015 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:09.199969    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:09.215004 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:09.214973    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:33:09.215159 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:09.215130    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 03:33:09.215203 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:09.215155    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:33:09.215203 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:09.215191    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:33:09.215277 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:09.215247    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert podName:ed31fb70-7e2b-4d84-946b-b00431c7e731 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:25.215230842 +0000 UTC m=+66.706226697 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9gjc5" (UID: "ed31fb70-7e2b-4d84-946b-b00431c7e731") : secret "networking-console-plugin-cert" not found
Jul 10 03:33:09.215277 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:09.215272    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:33:09.215353 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:09.215308    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 03:33:09.215353 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:09.215326    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f84d8746d-csqp6: secret "image-registry-tls" not found
Jul 10 03:33:09.215423 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:09.215316    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls podName:0843417a-dbe0-45e1-96c6-4510eb4d74ef nodeName:}" failed. No retries permitted until 2026-07-10 03:33:25.215303736 +0000 UTC m=+66.706299588 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls") pod "dns-default-4rqhk" (UID: "0843417a-dbe0-45e1-96c6-4510eb4d74ef") : secret "dns-default-metrics-tls" not found
Jul 10 03:33:09.215423 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:09.215386    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls podName:32888fa2-39e4-4de1-9a19-dee518a05139 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:25.215372353 +0000 UTC m=+66.706368211 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls") pod "image-registry-7f84d8746d-csqp6" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139") : secret "image-registry-tls" not found
Jul 10 03:33:20.232556 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:20.232512    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-knklw"
Jul 10 03:33:20.388458 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:20.388422    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:20.491753 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:20.491678    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:22.316353 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:22.316319    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:22.376644 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:22.376608    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:23.498324 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:23.498290    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:23.599181 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:23.599152    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:24.731762 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:24.731715    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:33:24.732175 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:24.731796    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:33:24.734131 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:24.734103    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 03:33:24.734200 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:24.734185    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 03:33:24.742141 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:24.742121    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 03:33:24.742222 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:24.742209    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs podName:776389fa-994a-4f3b-bcfb-48c486232e9b nodeName:}" failed. No retries permitted until 2026-07-10 03:34:28.742185997 +0000 UTC m=+130.233181853 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs") pod "network-metrics-daemon-p7l8g" (UID: "776389fa-994a-4f3b-bcfb-48c486232e9b") : secret "metrics-daemon-secret" not found
Jul 10 03:33:24.745522 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:24.745491    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed5febd9-f003-483e-bd13-02cf0eeb73b6-original-pull-secret\") pod \"global-pull-secret-syncer-lzrdd\" (UID: \"ed5febd9-f003-483e-bd13-02cf0eeb73b6\") " pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:33:24.832673 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:24.832608    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j68rf\" (UniqueName: \"kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf\") pod \"network-check-target-sphtj\" (UID: \"13ef3741-844f-4dc4-b748-7a96098484b8\") " pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:33:24.835080 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:24.835059    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 03:33:24.846258 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:24.846241    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 03:33:24.856541 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:24.856521    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j68rf\" (UniqueName: \"kubernetes.io/projected/13ef3741-844f-4dc4-b748-7a96098484b8-kube-api-access-j68rf\") pod \"network-check-target-sphtj\" (UID: \"13ef3741-844f-4dc4-b748-7a96098484b8\") " pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:33:25.001965 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:25.001890    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lzrdd"
Jul 10 03:33:25.019099 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:25.019064    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c4cxw\""
Jul 10 03:33:25.026987 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:25.026958    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:33:25.148461 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:25.148388    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lzrdd"]
Jul 10 03:33:25.153022 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:33:25.152994    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded5febd9_f003_483e_bd13_02cf0eeb73b6.slice/crio-61f4b7865f3d499d20d5d5c6caf2bf6bbca9c705521ff7122e5eb8181c1848b2 WatchSource:0}: Error finding container 61f4b7865f3d499d20d5d5c6caf2bf6bbca9c705521ff7122e5eb8181c1848b2: Status 404 returned error can't find the container with id 61f4b7865f3d499d20d5d5c6caf2bf6bbca9c705521ff7122e5eb8181c1848b2
Jul 10 03:33:25.175531 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:25.175503    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sphtj"]
Jul 10 03:33:25.178817 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:33:25.178791    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ef3741_844f_4dc4_b748_7a96098484b8.slice/crio-8ccb4df14f6ff401d013a2d78291e02f3a3da98eb82596d701f7c85d9e9bb4ea WatchSource:0}: Error finding container 8ccb4df14f6ff401d013a2d78291e02f3a3da98eb82596d701f7c85d9e9bb4ea: Status 404 returned error can't find the container with id 8ccb4df14f6ff401d013a2d78291e02f3a3da98eb82596d701f7c85d9e9bb4ea
Jul 10 03:33:25.234438 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:25.234402    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:33:25.234644 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:25.234448    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:33:25.234644 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:25.234498    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:33:25.234644 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:25.234575    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:33:25.234644 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:25.234607    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 03:33:25.234644 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:25.234622    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f84d8746d-csqp6: secret "image-registry-tls" not found
Jul 10 03:33:25.234644 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:25.234667    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls podName:0843417a-dbe0-45e1-96c6-4510eb4d74ef nodeName:}" failed. No retries permitted until 2026-07-10 03:33:57.234628023 +0000 UTC m=+98.725623896 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls") pod "dns-default-4rqhk" (UID: "0843417a-dbe0-45e1-96c6-4510eb4d74ef") : secret "dns-default-metrics-tls" not found
Jul 10 03:33:25.234914 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:25.234609    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 03:33:25.234914 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:25.234691    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls podName:32888fa2-39e4-4de1-9a19-dee518a05139 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:57.234679417 +0000 UTC m=+98.725675279 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls") pod "image-registry-7f84d8746d-csqp6" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139") : secret "image-registry-tls" not found
Jul 10 03:33:25.234914 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:25.234763    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert podName:ed31fb70-7e2b-4d84-946b-b00431c7e731 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:57.23473477 +0000 UTC m=+98.725730644 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9gjc5" (UID: "ed31fb70-7e2b-4d84-946b-b00431c7e731") : secret "networking-console-plugin-cert" not found
Jul 10 03:33:25.304176 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:25.304072    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lzrdd" event={"ID":"ed5febd9-f003-483e-bd13-02cf0eeb73b6","Type":"ContainerStarted","Data":"61f4b7865f3d499d20d5d5c6caf2bf6bbca9c705521ff7122e5eb8181c1848b2"}
Jul 10 03:33:25.305182 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:25.305152    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sphtj" event={"ID":"13ef3741-844f-4dc4-b748-7a96098484b8","Type":"ContainerStarted","Data":"8ccb4df14f6ff401d013a2d78291e02f3a3da98eb82596d701f7c85d9e9bb4ea"}
Jul 10 03:33:25.879600 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:25.879560    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:25.932219 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:25.932178    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:30.320418 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:30.320324    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sphtj" event={"ID":"13ef3741-844f-4dc4-b748-7a96098484b8","Type":"ContainerStarted","Data":"d1084f506f33bcef1b6573512587b24b667430ae85d596740607aad01a884633"}
Jul 10 03:33:30.320418 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:30.320391    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:33:30.321867 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:30.321846    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lzrdd" event={"ID":"ed5febd9-f003-483e-bd13-02cf0eeb73b6","Type":"ContainerStarted","Data":"960449f830abf745db1d8b553c4234c48a8dfe915cf26506becbbdb75b0faa33"}
Jul 10 03:33:30.335232 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:30.335187    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sphtj" podStartSLOduration=66.447482511 podStartE2EDuration="1m11.33517449s" totalImagesPullingTime="4.887691979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:30.334057099 +0000 UTC m=+71.825052988" watchObservedRunningTime="2026-07-10 03:33:30.33517449 +0000 UTC m=+71.826170363"
Jul 10 03:33:30.350326 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:30.348860    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lzrdd" podStartSLOduration=66.430901718 podStartE2EDuration="1m11.348838087s" totalImagesPullingTime="4.917936369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:30.347521117 +0000 UTC m=+71.838516990" watchObservedRunningTime="2026-07-10 03:33:30.348838087 +0000 UTC m=+71.839833961"
Jul 10 03:33:33.803832 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:33.803766    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:33.853741 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:33.853710    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:46.455381 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:46.455263    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:46.509273 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:46.509240    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:46.567495 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:46.567464    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:33:57.268186 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:57.268130    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:33:57.268186 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:57.268188    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:33:57.268720 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:57.268284    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:33:57.268720 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:33:57.268298    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:33:57.268720 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:57.268347    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls podName:0843417a-dbe0-45e1-96c6-4510eb4d74ef nodeName:}" failed. No retries permitted until 2026-07-10 03:35:01.268329109 +0000 UTC m=+162.759324964 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls") pod "dns-default-4rqhk" (UID: "0843417a-dbe0-45e1-96c6-4510eb4d74ef") : secret "dns-default-metrics-tls" not found
Jul 10 03:33:57.268720 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:57.268395    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 03:33:57.268720 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:57.268434    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 03:33:57.268720 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:57.268453    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f84d8746d-csqp6: secret "image-registry-tls" not found
Jul 10 03:33:57.268720 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:57.268445    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert podName:ed31fb70-7e2b-4d84-946b-b00431c7e731 nodeName:}" failed. No retries permitted until 2026-07-10 03:35:01.268434383 +0000 UTC m=+162.759430233 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9gjc5" (UID: "ed31fb70-7e2b-4d84-946b-b00431c7e731") : secret "networking-console-plugin-cert" not found
Jul 10 03:33:57.268720 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:33:57.268534    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls podName:32888fa2-39e4-4de1-9a19-dee518a05139 nodeName:}" failed. No retries permitted until 2026-07-10 03:35:01.268513017 +0000 UTC m=+162.759508887 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls") pod "image-registry-7f84d8746d-csqp6" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139") : secret "image-registry-tls" not found
Jul 10 03:34:01.325617 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:34:01.325585    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sphtj"
Jul 10 03:34:28.801911 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:34:28.801867    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:34:28.802377 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:34:28.802023    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 03:34:28.802377 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:34:28.802099    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs podName:776389fa-994a-4f3b-bcfb-48c486232e9b nodeName:}" failed. No retries permitted until 2026-07-10 03:36:30.802082531 +0000 UTC m=+252.293078385 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs") pod "network-metrics-daemon-p7l8g" (UID: "776389fa-994a-4f3b-bcfb-48c486232e9b") : secret "metrics-daemon-secret" not found
Jul 10 03:34:56.355385 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:34:56.355326    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5" podUID="ed31fb70-7e2b-4d84-946b-b00431c7e731"
Jul 10 03:34:56.363528 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:34:56.363498    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6" podUID="32888fa2-39e4-4de1-9a19-dee518a05139"
Jul 10 03:34:56.377814 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:34:56.377781    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4rqhk" podUID="0843417a-dbe0-45e1-96c6-4510eb4d74ef"
Jul 10 03:34:56.518986 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:34:56.518957    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:34:56.519116 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:34:56.518957    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:34:56.519155 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:34:56.518957    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:34:58.009973 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:34:58.009921    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:35:01.328467 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:35:01.328430    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:35:01.329030 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:35:01.328476    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:35:01.329030 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:35:01.328528    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:35:01.329030 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:35:01.328573    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:35:01.329030 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:35:01.328637    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 03:35:01.329030 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:35:01.328674    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f84d8746d-csqp6: secret "image-registry-tls" not found
Jul 10 03:35:01.329030 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:35:01.328640    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls podName:0843417a-dbe0-45e1-96c6-4510eb4d74ef nodeName:}" failed. No retries permitted until 2026-07-10 03:37:03.328621664 +0000 UTC m=+284.819617517 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls") pod "dns-default-4rqhk" (UID: "0843417a-dbe0-45e1-96c6-4510eb4d74ef") : secret "dns-default-metrics-tls" not found
Jul 10 03:35:01.329030 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:35:01.328640    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 03:35:01.329030 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:35:01.328737    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls podName:32888fa2-39e4-4de1-9a19-dee518a05139 nodeName:}" failed. No retries permitted until 2026-07-10 03:37:03.328724261 +0000 UTC m=+284.819720116 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls") pod "image-registry-7f84d8746d-csqp6" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139") : secret "image-registry-tls" not found
Jul 10 03:35:01.329030 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:35:01.328804    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert podName:ed31fb70-7e2b-4d84-946b-b00431c7e731 nodeName:}" failed. No retries permitted until 2026-07-10 03:37:03.32878964 +0000 UTC m=+284.819785491 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9gjc5" (UID: "ed31fb70-7e2b-4d84-946b-b00431c7e731") : secret "networking-console-plugin-cert" not found
Jul 10 03:35:10.988556 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:35:10.988461    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:36:24.384624 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.384588    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4b5xs"]
Jul 10 03:36:24.387373 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.387357    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4b5xs"
Jul 10 03:36:24.389883 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.389862    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 10 03:36:24.390126 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.390107    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 10 03:36:24.391144 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.391123    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-xdnxn\""
Jul 10 03:36:24.391835 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.391817    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-nkxvw"]
Jul 10 03:36:24.394626 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.394610    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-l22sz"]
Jul 10 03:36:24.394769 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.394749    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nkxvw"
Jul 10 03:36:24.396978 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.396955    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-ppmcj\""
Jul 10 03:36:24.397229 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.397207    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-dcjt9"]
Jul 10 03:36:24.397339 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.397328    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.400877 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.400858    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4b5xs"]
Jul 10 03:36:24.400987 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.400951    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.401544 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.401506    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 10 03:36:24.401544 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.401528    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 03:36:24.403256 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.403220    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 03:36:24.403496 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.403472    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-h8khf\""
Jul 10 03:36:24.410009 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.409746    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 10 03:36:24.410009 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.409981    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 10 03:36:24.410201 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.410031    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 10 03:36:24.410201 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.410036    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 10 03:36:24.410201 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.409979    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 10 03:36:24.411460 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.410418    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-p5sbh\""
Jul 10 03:36:24.412952 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.412933    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-nkxvw"]
Jul 10 03:36:24.414155 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.414131    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-l22sz"]
Jul 10 03:36:24.414555 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.414538    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 10 03:36:24.415290 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.415274    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 10 03:36:24.422405 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.422384    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-dcjt9"]
Jul 10 03:36:24.434013 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.433989    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b41fb4a3-4b8d-4c2d-b751-2283a10b4234-serving-cert\") pod \"console-operator-7866497494-dcjt9\" (UID: \"b41fb4a3-4b8d-4c2d-b751-2283a10b4234\") " pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.434100 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.434025    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tc7d\" (UniqueName: \"kubernetes.io/projected/f96f84ef-4688-4217-9a4a-2347d1e3ba13-kube-api-access-4tc7d\") pod \"network-check-source-869dcfcdcd-nkxvw\" (UID: \"f96f84ef-4688-4217-9a4a-2347d1e3ba13\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nkxvw"
Jul 10 03:36:24.434100 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.434043    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfm5q\" (UniqueName: \"kubernetes.io/projected/b858fbad-d7b2-4b25-b556-c852059bdef7-kube-api-access-rfm5q\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.434100 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.434059    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b858fbad-d7b2-4b25-b556-c852059bdef7-tmp\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.434218 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.434142    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b858fbad-d7b2-4b25-b556-c852059bdef7-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.434218 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.434195    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b41fb4a3-4b8d-4c2d-b751-2283a10b4234-config\") pod \"console-operator-7866497494-dcjt9\" (UID: \"b41fb4a3-4b8d-4c2d-b751-2283a10b4234\") " pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.434281 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.434232    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b858fbad-d7b2-4b25-b556-c852059bdef7-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.434281 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.434247    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b858fbad-d7b2-4b25-b556-c852059bdef7-snapshots\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.434281 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.434267    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjcd9\" (UniqueName: \"kubernetes.io/projected/b41fb4a3-4b8d-4c2d-b751-2283a10b4234-kube-api-access-hjcd9\") pod \"console-operator-7866497494-dcjt9\" (UID: \"b41fb4a3-4b8d-4c2d-b751-2283a10b4234\") " pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.434405 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.434295    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9rgm\" (UniqueName: \"kubernetes.io/projected/05b51b6d-8448-4e78-9231-292041790e0f-kube-api-access-q9rgm\") pod \"volume-data-source-validator-78495597f6-4b5xs\" (UID: \"05b51b6d-8448-4e78-9231-292041790e0f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4b5xs"
Jul 10 03:36:24.434405 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.434320    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b858fbad-d7b2-4b25-b556-c852059bdef7-serving-cert\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.434405 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.434336    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b41fb4a3-4b8d-4c2d-b751-2283a10b4234-trusted-ca\") pod \"console-operator-7866497494-dcjt9\" (UID: \"b41fb4a3-4b8d-4c2d-b751-2283a10b4234\") " pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.488903 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.488858    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"]
Jul 10 03:36:24.491839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.491816    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"
Jul 10 03:36:24.494239 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.494219    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 10 03:36:24.494469 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.494455    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-k29tw\""
Jul 10 03:36:24.494887 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.494868    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 10 03:36:24.495210 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.495195    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 10 03:36:24.495287 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.495214    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 10 03:36:24.501745 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.501726    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"]
Jul 10 03:36:24.535691 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.535662    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9rgm\" (UniqueName: \"kubernetes.io/projected/05b51b6d-8448-4e78-9231-292041790e0f-kube-api-access-q9rgm\") pod \"volume-data-source-validator-78495597f6-4b5xs\" (UID: \"05b51b6d-8448-4e78-9231-292041790e0f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4b5xs"
Jul 10 03:36:24.535691 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.535692    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b858fbad-d7b2-4b25-b556-c852059bdef7-serving-cert\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.535907 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.535713    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b41fb4a3-4b8d-4c2d-b751-2283a10b4234-trusted-ca\") pod \"console-operator-7866497494-dcjt9\" (UID: \"b41fb4a3-4b8d-4c2d-b751-2283a10b4234\") " pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.535907 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.535734    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b41fb4a3-4b8d-4c2d-b751-2283a10b4234-serving-cert\") pod \"console-operator-7866497494-dcjt9\" (UID: \"b41fb4a3-4b8d-4c2d-b751-2283a10b4234\") " pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.535907 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.535757    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5x8q\" (UniqueName: \"kubernetes.io/projected/6c312753-450d-4a92-822e-4638a7eafc6f-kube-api-access-z5x8q\") pod \"kube-storage-version-migrator-operator-799cb7f958-j6bzq\" (UID: \"6c312753-450d-4a92-822e-4638a7eafc6f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"
Jul 10 03:36:24.535907 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.535792    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tc7d\" (UniqueName: \"kubernetes.io/projected/f96f84ef-4688-4217-9a4a-2347d1e3ba13-kube-api-access-4tc7d\") pod \"network-check-source-869dcfcdcd-nkxvw\" (UID: \"f96f84ef-4688-4217-9a4a-2347d1e3ba13\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nkxvw"
Jul 10 03:36:24.535907 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.535811    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c312753-450d-4a92-822e-4638a7eafc6f-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-j6bzq\" (UID: \"6c312753-450d-4a92-822e-4638a7eafc6f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"
Jul 10 03:36:24.535907 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.535836    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfm5q\" (UniqueName: \"kubernetes.io/projected/b858fbad-d7b2-4b25-b556-c852059bdef7-kube-api-access-rfm5q\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.535907 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.535863    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b858fbad-d7b2-4b25-b556-c852059bdef7-tmp\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.536279 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.535943    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c312753-450d-4a92-822e-4638a7eafc6f-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-j6bzq\" (UID: \"6c312753-450d-4a92-822e-4638a7eafc6f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"
Jul 10 03:36:24.536279 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.536000    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b858fbad-d7b2-4b25-b556-c852059bdef7-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.536279 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.536079    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b41fb4a3-4b8d-4c2d-b751-2283a10b4234-config\") pod \"console-operator-7866497494-dcjt9\" (UID: \"b41fb4a3-4b8d-4c2d-b751-2283a10b4234\") " pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.536279 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.536145    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b858fbad-d7b2-4b25-b556-c852059bdef7-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.536279 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.536175    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b858fbad-d7b2-4b25-b556-c852059bdef7-snapshots\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.536279 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.536225    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjcd9\" (UniqueName: \"kubernetes.io/projected/b41fb4a3-4b8d-4c2d-b751-2283a10b4234-kube-api-access-hjcd9\") pod \"console-operator-7866497494-dcjt9\" (UID: \"b41fb4a3-4b8d-4c2d-b751-2283a10b4234\") " pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.536601 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.536288    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b858fbad-d7b2-4b25-b556-c852059bdef7-tmp\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.536746 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.536727    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b41fb4a3-4b8d-4c2d-b751-2283a10b4234-trusted-ca\") pod \"console-operator-7866497494-dcjt9\" (UID: \"b41fb4a3-4b8d-4c2d-b751-2283a10b4234\") " pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.536806 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.536780    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b41fb4a3-4b8d-4c2d-b751-2283a10b4234-config\") pod \"console-operator-7866497494-dcjt9\" (UID: \"b41fb4a3-4b8d-4c2d-b751-2283a10b4234\") " pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.536916 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.536893    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b858fbad-d7b2-4b25-b556-c852059bdef7-snapshots\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.536993 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.536977    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b858fbad-d7b2-4b25-b556-c852059bdef7-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.537224 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.537204    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b858fbad-d7b2-4b25-b556-c852059bdef7-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.538211 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.538189    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b858fbad-d7b2-4b25-b556-c852059bdef7-serving-cert\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.538743 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.538727    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b41fb4a3-4b8d-4c2d-b751-2283a10b4234-serving-cert\") pod \"console-operator-7866497494-dcjt9\" (UID: \"b41fb4a3-4b8d-4c2d-b751-2283a10b4234\") " pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.544745 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.544716    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9rgm\" (UniqueName: \"kubernetes.io/projected/05b51b6d-8448-4e78-9231-292041790e0f-kube-api-access-q9rgm\") pod \"volume-data-source-validator-78495597f6-4b5xs\" (UID: \"05b51b6d-8448-4e78-9231-292041790e0f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4b5xs"
Jul 10 03:36:24.545119 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.545089    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfm5q\" (UniqueName: \"kubernetes.io/projected/b858fbad-d7b2-4b25-b556-c852059bdef7-kube-api-access-rfm5q\") pod \"insights-operator-68d9dd94cf-l22sz\" (UID: \"b858fbad-d7b2-4b25-b556-c852059bdef7\") " pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.545216 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.545194    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjcd9\" (UniqueName: \"kubernetes.io/projected/b41fb4a3-4b8d-4c2d-b751-2283a10b4234-kube-api-access-hjcd9\") pod \"console-operator-7866497494-dcjt9\" (UID: \"b41fb4a3-4b8d-4c2d-b751-2283a10b4234\") " pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.545557 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.545535    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tc7d\" (UniqueName: \"kubernetes.io/projected/f96f84ef-4688-4217-9a4a-2347d1e3ba13-kube-api-access-4tc7d\") pod \"network-check-source-869dcfcdcd-nkxvw\" (UID: \"f96f84ef-4688-4217-9a4a-2347d1e3ba13\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nkxvw"
Jul 10 03:36:24.637038 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.636956    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5x8q\" (UniqueName: \"kubernetes.io/projected/6c312753-450d-4a92-822e-4638a7eafc6f-kube-api-access-z5x8q\") pod \"kube-storage-version-migrator-operator-799cb7f958-j6bzq\" (UID: \"6c312753-450d-4a92-822e-4638a7eafc6f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"
Jul 10 03:36:24.637197 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.636994    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c312753-450d-4a92-822e-4638a7eafc6f-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-j6bzq\" (UID: \"6c312753-450d-4a92-822e-4638a7eafc6f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"
Jul 10 03:36:24.637197 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.637160    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c312753-450d-4a92-822e-4638a7eafc6f-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-j6bzq\" (UID: \"6c312753-450d-4a92-822e-4638a7eafc6f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"
Jul 10 03:36:24.637589 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.637560    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c312753-450d-4a92-822e-4638a7eafc6f-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-j6bzq\" (UID: \"6c312753-450d-4a92-822e-4638a7eafc6f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"
Jul 10 03:36:24.639702 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.639679    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c312753-450d-4a92-822e-4638a7eafc6f-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-j6bzq\" (UID: \"6c312753-450d-4a92-822e-4638a7eafc6f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"
Jul 10 03:36:24.643978 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.643956    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:24.644661 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.644613    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5x8q\" (UniqueName: \"kubernetes.io/projected/6c312753-450d-4a92-822e-4638a7eafc6f-kube-api-access-z5x8q\") pod \"kube-storage-version-migrator-operator-799cb7f958-j6bzq\" (UID: \"6c312753-450d-4a92-822e-4638a7eafc6f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"
Jul 10 03:36:24.696840 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.696798    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4b5xs"
Jul 10 03:36:24.707955 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.707925    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nkxvw"
Jul 10 03:36:24.716074 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.716051    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-l22sz"
Jul 10 03:36:24.721110 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.721083    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:24.732178 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.732112    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:24.803122 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.802864    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"
Jul 10 03:36:24.851781 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.851748    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4b5xs"]
Jul 10 03:36:24.856118 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:36:24.856068    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05b51b6d_8448_4e78_9231_292041790e0f.slice/crio-b4809ecad87161af75fcc4144c011d0dd6ad4bae9a1b324a6f95dbfd5a07efaf WatchSource:0}: Error finding container b4809ecad87161af75fcc4144c011d0dd6ad4bae9a1b324a6f95dbfd5a07efaf: Status 404 returned error can't find the container with id b4809ecad87161af75fcc4144c011d0dd6ad4bae9a1b324a6f95dbfd5a07efaf
Jul 10 03:36:24.873730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.873634    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-nkxvw"]
Jul 10 03:36:24.878138 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:36:24.878083    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf96f84ef_4688_4217_9a4a_2347d1e3ba13.slice/crio-8d812a9572012a668edeeea5433a185055f98f7b93011aeffc1286d8ffd5d980 WatchSource:0}: Error finding container 8d812a9572012a668edeeea5433a185055f98f7b93011aeffc1286d8ffd5d980: Status 404 returned error can't find the container with id 8d812a9572012a668edeeea5433a185055f98f7b93011aeffc1286d8ffd5d980
Jul 10 03:36:24.952093 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:24.952069    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq"]
Jul 10 03:36:24.954399 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:36:24.954374    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c312753_450d_4a92_822e_4638a7eafc6f.slice/crio-5c9d49563e824ad96874b0e3768bcf31ec9efd3edcdfe449587152eefd9e2def WatchSource:0}: Error finding container 5c9d49563e824ad96874b0e3768bcf31ec9efd3edcdfe449587152eefd9e2def: Status 404 returned error can't find the container with id 5c9d49563e824ad96874b0e3768bcf31ec9efd3edcdfe449587152eefd9e2def
Jul 10 03:36:25.098804 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:25.098776    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-l22sz"]
Jul 10 03:36:25.102253 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:25.102226    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-dcjt9"]
Jul 10 03:36:25.102366 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:36:25.102269    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb858fbad_d7b2_4b25_b556_c852059bdef7.slice/crio-9269a13a9356d75c8d4c925bbf80137572bf6efe451158cac5fb6e9e8d2da998 WatchSource:0}: Error finding container 9269a13a9356d75c8d4c925bbf80137572bf6efe451158cac5fb6e9e8d2da998: Status 404 returned error can't find the container with id 9269a13a9356d75c8d4c925bbf80137572bf6efe451158cac5fb6e9e8d2da998
Jul 10 03:36:25.104509 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:36:25.104489    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41fb4a3_4b8d_4c2d_b751_2283a10b4234.slice/crio-a251d64886dbd0b1f1bb0b4f6d46cdccfa44b95e330fe3f145f6457b50827f7d WatchSource:0}: Error finding container a251d64886dbd0b1f1bb0b4f6d46cdccfa44b95e330fe3f145f6457b50827f7d: Status 404 returned error can't find the container with id a251d64886dbd0b1f1bb0b4f6d46cdccfa44b95e330fe3f145f6457b50827f7d
Jul 10 03:36:25.735429 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:25.735358    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq" event={"ID":"6c312753-450d-4a92-822e-4638a7eafc6f","Type":"ContainerStarted","Data":"5c9d49563e824ad96874b0e3768bcf31ec9efd3edcdfe449587152eefd9e2def"}
Jul 10 03:36:25.738239 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:25.738203    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-dcjt9" event={"ID":"b41fb4a3-4b8d-4c2d-b751-2283a10b4234","Type":"ContainerStarted","Data":"a251d64886dbd0b1f1bb0b4f6d46cdccfa44b95e330fe3f145f6457b50827f7d"}
Jul 10 03:36:25.739723 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:25.739672    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4b5xs" event={"ID":"05b51b6d-8448-4e78-9231-292041790e0f","Type":"ContainerStarted","Data":"b4809ecad87161af75fcc4144c011d0dd6ad4bae9a1b324a6f95dbfd5a07efaf"}
Jul 10 03:36:25.742587 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:25.742560    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nkxvw" event={"ID":"f96f84ef-4688-4217-9a4a-2347d1e3ba13","Type":"ContainerStarted","Data":"27a3eb2c916b329005c668e2a83fd7a805cd2b4a1ab36f4754d856d85d60425f"}
Jul 10 03:36:25.742703 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:25.742607    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nkxvw" event={"ID":"f96f84ef-4688-4217-9a4a-2347d1e3ba13","Type":"ContainerStarted","Data":"8d812a9572012a668edeeea5433a185055f98f7b93011aeffc1286d8ffd5d980"}
Jul 10 03:36:25.744519 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:25.744467    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-l22sz" event={"ID":"b858fbad-d7b2-4b25-b556-c852059bdef7","Type":"ContainerStarted","Data":"9269a13a9356d75c8d4c925bbf80137572bf6efe451158cac5fb6e9e8d2da998"}
Jul 10 03:36:25.760670 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:25.759323    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nkxvw" podStartSLOduration=1.759304394 podStartE2EDuration="1.759304394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:36:25.756765325 +0000 UTC m=+247.247761195" watchObservedRunningTime="2026-07-10 03:36:25.759304394 +0000 UTC m=+247.250300269"
Jul 10 03:36:26.748847 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:26.748777    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4b5xs" event={"ID":"05b51b6d-8448-4e78-9231-292041790e0f","Type":"ContainerStarted","Data":"ae2327c03a182b3fbce57b98ee7288cab6706835d8932940561b622a14f64c0b"}
Jul 10 03:36:26.765079 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:26.765019    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4b5xs" podStartSLOduration=1.5043363859999999 podStartE2EDuration="2.7649998s" totalImagesPullingTime="1.260663414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:36:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:36:26.764430505 +0000 UTC m=+248.255426379" watchObservedRunningTime="2026-07-10 03:36:26.7649998 +0000 UTC m=+248.255995677"
Jul 10 03:36:26.837378 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:26.837342    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:26.886911 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:26.886870    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:28.468452 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:28.468415    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:28.757200 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:28.757106    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq" event={"ID":"6c312753-450d-4a92-822e-4638a7eafc6f","Type":"ContainerStarted","Data":"16f7fd2d482454ccae32f7decb8662b1b7461a11420e212ca321d4b8daa5ee6e"}
Jul 10 03:36:28.758873 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:28.758848    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/0.log"
Jul 10 03:36:28.759001 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:28.758892    2561 generic.go:356] "Generic (PLEG): container finished" podID="b41fb4a3-4b8d-4c2d-b751-2283a10b4234" containerID="3434c7f741038bb7155b6a5fd955e44637404d05e3e7dcca84f50b4945acafef" exitCode=255
Jul 10 03:36:28.759055 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:28.758989    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-dcjt9" event={"ID":"b41fb4a3-4b8d-4c2d-b751-2283a10b4234","Type":"ContainerDied","Data":"3434c7f741038bb7155b6a5fd955e44637404d05e3e7dcca84f50b4945acafef"}
Jul 10 03:36:28.759151 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:28.759132    2561 scope.go:117] "RemoveContainer" containerID="3434c7f741038bb7155b6a5fd955e44637404d05e3e7dcca84f50b4945acafef"
Jul 10 03:36:28.764034 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:28.764014    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-l22sz" event={"ID":"b858fbad-d7b2-4b25-b556-c852059bdef7","Type":"ContainerStarted","Data":"8a93a685948463c31d09c69351d4889b982610c90a3c819b087261a8a3fc9240"}
Jul 10 03:36:28.773501 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:28.773454    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq" podStartSLOduration=1.706985631 podStartE2EDuration="4.773436274s" totalImagesPullingTime="3.066450643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:36:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:36:28.77185379 +0000 UTC m=+250.262849687" watchObservedRunningTime="2026-07-10 03:36:28.773436274 +0000 UTC m=+250.264432127"
Jul 10 03:36:28.896164 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:28.896136    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:28.996493 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:28.996460    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:29.111313 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.111225    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-l22sz" podStartSLOduration=2.188763337 podStartE2EDuration="5.111203755s" totalImagesPullingTime="2.922440418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:36:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:36:28.79962425 +0000 UTC m=+250.290620126" watchObservedRunningTime="2026-07-10 03:36:29.111203755 +0000 UTC m=+250.602199629"
Jul 10 03:36:29.112224 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.112202    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:29.112319 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.112297    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xtcdb"]
Jul 10 03:36:29.115288 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.115273    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xtcdb"
Jul 10 03:36:29.119818 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.118012    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-knfjr\""
Jul 10 03:36:29.119818 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.118309    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 10 03:36:29.119818 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.118568    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 10 03:36:29.125042 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.125018    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xtcdb"]
Jul 10 03:36:29.178407 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.178371    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nsl5\" (UniqueName: \"kubernetes.io/projected/fbb60d0f-3458-4983-baa7-905512ac72d8-kube-api-access-4nsl5\") pod \"migrator-7f59b54f6c-xtcdb\" (UID: \"fbb60d0f-3458-4983-baa7-905512ac72d8\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xtcdb"
Jul 10 03:36:29.217333 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.217299    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:29.279364 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.279321    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nsl5\" (UniqueName: \"kubernetes.io/projected/fbb60d0f-3458-4983-baa7-905512ac72d8-kube-api-access-4nsl5\") pod \"migrator-7f59b54f6c-xtcdb\" (UID: \"fbb60d0f-3458-4983-baa7-905512ac72d8\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xtcdb"
Jul 10 03:36:29.285699 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.285667    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tnvwg"]
Jul 10 03:36:29.289215 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.289152    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nsl5\" (UniqueName: \"kubernetes.io/projected/fbb60d0f-3458-4983-baa7-905512ac72d8-kube-api-access-4nsl5\") pod \"migrator-7f59b54f6c-xtcdb\" (UID: \"fbb60d0f-3458-4983-baa7-905512ac72d8\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xtcdb"
Jul 10 03:36:29.290028 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.290012    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.292305 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.292283    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 03:36:29.292766 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.292749    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lwzwp\""
Jul 10 03:36:29.293400 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.293235    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 03:36:29.299366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.299330    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:29.300003 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.299977    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tnvwg"]
Jul 10 03:36:29.352865 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.352833    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:29.380090 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.380059    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.380245 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.380096    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.380245 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.380138    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6gwc\" (UniqueName: \"kubernetes.io/projected/a2b5f043-d820-4234-a832-021b63224ee9-kube-api-access-w6gwc\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.380245 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.380177    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2b5f043-d820-4234-a832-021b63224ee9-data-volume\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.380245 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.380211    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a2b5f043-d820-4234-a832-021b63224ee9-crio-socket\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.430242 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.430209    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xtcdb"
Jul 10 03:36:29.480639 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.480605    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.481023 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.480697    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.481023 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.480753    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6gwc\" (UniqueName: \"kubernetes.io/projected/a2b5f043-d820-4234-a832-021b63224ee9-kube-api-access-w6gwc\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.481023 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:29.480764    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 03:36:29.481023 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:29.480774    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:29.481023 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.480794    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2b5f043-d820-4234-a832-021b63224ee9-data-volume\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.481023 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:29.480835    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm podName:a2b5f043-d820-4234-a832-021b63224ee9 nodeName:}" failed. No retries permitted until 2026-07-10 03:36:29.980814939 +0000 UTC m=+251.471810807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnvwg" (UID: "a2b5f043-d820-4234-a832-021b63224ee9") : configmap "kube-rbac-proxy" not found
Jul 10 03:36:29.481341 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:29.481039    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls podName:a2b5f043-d820-4234-a832-021b63224ee9 nodeName:}" failed. No retries permitted until 2026-07-10 03:36:29.981017949 +0000 UTC m=+251.472013804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnvwg" (UID: "a2b5f043-d820-4234-a832-021b63224ee9") : secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:29.481341 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.481087    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a2b5f043-d820-4234-a832-021b63224ee9-crio-socket\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.481341 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.481197    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2b5f043-d820-4234-a832-021b63224ee9-data-volume\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.481341 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.481232    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a2b5f043-d820-4234-a832-021b63224ee9-crio-socket\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.488670 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.488626    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6gwc\" (UniqueName: \"kubernetes.io/projected/a2b5f043-d820-4234-a832-021b63224ee9-kube-api-access-w6gwc\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.552609 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.552535    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xtcdb"]
Jul 10 03:36:29.555731 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:36:29.555703    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbb60d0f_3458_4983_baa7_905512ac72d8.slice/crio-26e9b0dbba13c06ee966e310f00579eee7fee2fc5cdd5d3955e154852d6fd5f8 WatchSource:0}: Error finding container 26e9b0dbba13c06ee966e310f00579eee7fee2fc5cdd5d3955e154852d6fd5f8: Status 404 returned error can't find the container with id 26e9b0dbba13c06ee966e310f00579eee7fee2fc5cdd5d3955e154852d6fd5f8
Jul 10 03:36:29.769115 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.769089    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:36:29.769518 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.769502    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/0.log"
Jul 10 03:36:29.769590 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.769535    2561 generic.go:356] "Generic (PLEG): container finished" podID="b41fb4a3-4b8d-4c2d-b751-2283a10b4234" containerID="b7835bfe54622b89302a08d8317cbf2f9eb70ead481f6aa42265a982d51dabc8" exitCode=255
Jul 10 03:36:29.769623 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.769599    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-dcjt9" event={"ID":"b41fb4a3-4b8d-4c2d-b751-2283a10b4234","Type":"ContainerDied","Data":"b7835bfe54622b89302a08d8317cbf2f9eb70ead481f6aa42265a982d51dabc8"}
Jul 10 03:36:29.769696 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.769682    2561 scope.go:117] "RemoveContainer" containerID="3434c7f741038bb7155b6a5fd955e44637404d05e3e7dcca84f50b4945acafef"
Jul 10 03:36:29.769893 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.769873    2561 scope.go:117] "RemoveContainer" containerID="b7835bfe54622b89302a08d8317cbf2f9eb70ead481f6aa42265a982d51dabc8"
Jul 10 03:36:29.770094 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:29.770067    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-dcjt9_openshift-console-operator(b41fb4a3-4b8d-4c2d-b751-2283a10b4234)\"" pod="openshift-console-operator/console-operator-7866497494-dcjt9" podUID="b41fb4a3-4b8d-4c2d-b751-2283a10b4234"
Jul 10 03:36:29.770797 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.770741    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xtcdb" event={"ID":"fbb60d0f-3458-4983-baa7-905512ac72d8","Type":"ContainerStarted","Data":"26e9b0dbba13c06ee966e310f00579eee7fee2fc5cdd5d3955e154852d6fd5f8"}
Jul 10 03:36:29.986503 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.986462    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.986706 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:29.986592    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 03:36:29.986706 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:29.986593    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:29.986706 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:29.986688    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm podName:a2b5f043-d820-4234-a832-021b63224ee9 nodeName:}" failed. No retries permitted until 2026-07-10 03:36:30.986668586 +0000 UTC m=+252.477664451 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnvwg" (UID: "a2b5f043-d820-4234-a832-021b63224ee9") : configmap "kube-rbac-proxy" not found
Jul 10 03:36:29.986818 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:29.986688    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:29.986818 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:29.986785    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls podName:a2b5f043-d820-4234-a832-021b63224ee9 nodeName:}" failed. No retries permitted until 2026-07-10 03:36:30.986769101 +0000 UTC m=+252.477764963 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnvwg" (UID: "a2b5f043-d820-4234-a832-021b63224ee9") : secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:30.758198 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.758079    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-w4vh9"]
Jul 10 03:36:30.762441 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.762421    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9"
Jul 10 03:36:30.764612 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.764588    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 10 03:36:30.764798 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.764691    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 10 03:36:30.765548 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.765534    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 10 03:36:30.765631 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.765615    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-c6ds6\""
Jul 10 03:36:30.765796 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.765781    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 10 03:36:30.770122 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.770023    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-w4vh9"]
Jul 10 03:36:30.776805 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.776710    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:36:30.777080 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.777063    2561 scope.go:117] "RemoveContainer" containerID="b7835bfe54622b89302a08d8317cbf2f9eb70ead481f6aa42265a982d51dabc8"
Jul 10 03:36:30.777301 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:30.777235    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-dcjt9_openshift-console-operator(b41fb4a3-4b8d-4c2d-b751-2283a10b4234)\"" pod="openshift-console-operator/console-operator-7866497494-dcjt9" podUID="b41fb4a3-4b8d-4c2d-b751-2283a10b4234"
Jul 10 03:36:30.778579 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.778543    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xtcdb" event={"ID":"fbb60d0f-3458-4983-baa7-905512ac72d8","Type":"ContainerStarted","Data":"f73251c34baeef612213bc420edb92797547e0e20428aad60d2665e137fe67d1"}
Jul 10 03:36:30.792131 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.792087    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:30.892451 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.892409    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:30.894606 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.894583    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:36:30.894753 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.894614    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a3c97055-ca89-454c-a482-d39a5b67060b-signing-key\") pod \"service-ca-79f68b9f4f-w4vh9\" (UID: \"a3c97055-ca89-454c-a482-d39a5b67060b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9"
Jul 10 03:36:30.894827 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:30.894761    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 03:36:30.894879 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:30.894827    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs podName:776389fa-994a-4f3b-bcfb-48c486232e9b nodeName:}" failed. No retries permitted until 2026-07-10 03:38:32.89480855 +0000 UTC m=+374.385804405 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs") pod "network-metrics-daemon-p7l8g" (UID: "776389fa-994a-4f3b-bcfb-48c486232e9b") : secret "metrics-daemon-secret" not found
Jul 10 03:36:30.894942 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.894875    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a3c97055-ca89-454c-a482-d39a5b67060b-signing-cabundle\") pod \"service-ca-79f68b9f4f-w4vh9\" (UID: \"a3c97055-ca89-454c-a482-d39a5b67060b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9"
Jul 10 03:36:30.894942 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.894918    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cql2j\" (UniqueName: \"kubernetes.io/projected/a3c97055-ca89-454c-a482-d39a5b67060b-kube-api-access-cql2j\") pod \"service-ca-79f68b9f4f-w4vh9\" (UID: \"a3c97055-ca89-454c-a482-d39a5b67060b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9"
Jul 10 03:36:30.996383 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.996307    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a3c97055-ca89-454c-a482-d39a5b67060b-signing-cabundle\") pod \"service-ca-79f68b9f4f-w4vh9\" (UID: \"a3c97055-ca89-454c-a482-d39a5b67060b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9"
Jul 10 03:36:30.996383 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.996344    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cql2j\" (UniqueName: \"kubernetes.io/projected/a3c97055-ca89-454c-a482-d39a5b67060b-kube-api-access-cql2j\") pod \"service-ca-79f68b9f4f-w4vh9\" (UID: \"a3c97055-ca89-454c-a482-d39a5b67060b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9"
Jul 10 03:36:30.996547 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.996413    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a3c97055-ca89-454c-a482-d39a5b67060b-signing-key\") pod \"service-ca-79f68b9f4f-w4vh9\" (UID: \"a3c97055-ca89-454c-a482-d39a5b67060b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9"
Jul 10 03:36:30.996547 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.996447    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:30.996547 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.996482    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:30.996637 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:30.996561    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 03:36:30.996637 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:30.996598    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:30.996717 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:30.996642    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm podName:a2b5f043-d820-4234-a832-021b63224ee9 nodeName:}" failed. No retries permitted until 2026-07-10 03:36:32.996619282 +0000 UTC m=+254.487615136 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnvwg" (UID: "a2b5f043-d820-4234-a832-021b63224ee9") : configmap "kube-rbac-proxy" not found
Jul 10 03:36:30.996717 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:30.996687    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls podName:a2b5f043-d820-4234-a832-021b63224ee9 nodeName:}" failed. No retries permitted until 2026-07-10 03:36:32.996676766 +0000 UTC m=+254.487672617 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnvwg" (UID: "a2b5f043-d820-4234-a832-021b63224ee9") : secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:30.997088 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.997069    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a3c97055-ca89-454c-a482-d39a5b67060b-signing-cabundle\") pod \"service-ca-79f68b9f4f-w4vh9\" (UID: \"a3c97055-ca89-454c-a482-d39a5b67060b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9"
Jul 10 03:36:30.998755 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:30.998738    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a3c97055-ca89-454c-a482-d39a5b67060b-signing-key\") pod \"service-ca-79f68b9f4f-w4vh9\" (UID: \"a3c97055-ca89-454c-a482-d39a5b67060b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9"
Jul 10 03:36:31.004038 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:31.004017    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cql2j\" (UniqueName: \"kubernetes.io/projected/a3c97055-ca89-454c-a482-d39a5b67060b-kube-api-access-cql2j\") pod \"service-ca-79f68b9f4f-w4vh9\" (UID: \"a3c97055-ca89-454c-a482-d39a5b67060b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9"
Jul 10 03:36:31.081361 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:31.081327    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9"
Jul 10 03:36:31.201191 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:31.201158    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-w4vh9"]
Jul 10 03:36:31.204868 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:36:31.204824    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c97055_ca89_454c_a482_d39a5b67060b.slice/crio-c51279436150dd592e614942fc7be14f03c44c90c91f3341ce1bbf2067bc5c79 WatchSource:0}: Error finding container c51279436150dd592e614942fc7be14f03c44c90c91f3341ce1bbf2067bc5c79: Status 404 returned error can't find the container with id c51279436150dd592e614942fc7be14f03c44c90c91f3341ce1bbf2067bc5c79
Jul 10 03:36:31.629618 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:31.629583    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:31.783659 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:31.783606    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9" event={"ID":"a3c97055-ca89-454c-a482-d39a5b67060b","Type":"ContainerStarted","Data":"c51279436150dd592e614942fc7be14f03c44c90c91f3341ce1bbf2067bc5c79"}
Jul 10 03:36:31.785670 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:31.785624    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xtcdb" event={"ID":"fbb60d0f-3458-4983-baa7-905512ac72d8","Type":"ContainerStarted","Data":"4a000720974d96f0495afcb1978e476ad0dd806ab73df781d2c3edfd1d873a96"}
Jul 10 03:36:31.804077 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:31.804027    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xtcdb" podStartSLOduration=1.6927978700000001 podStartE2EDuration="2.804008345s" totalImagesPullingTime="1.111210475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:36:31.803263998 +0000 UTC m=+253.294259896" watchObservedRunningTime="2026-07-10 03:36:31.804008345 +0000 UTC m=+253.295004223"
Jul 10 03:36:32.079709 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:32.079675    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gz9tm_7aec8603-2c61-4c91-936d-728b9f3c641f/dns-node-resolver/0.log"
Jul 10 03:36:32.501835 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:32.501727    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:32.601491 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:32.601458    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:32.880019 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:32.879983    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4h4m7_4399c5c6-6b1b-4c30-bed8-d7ae494e282b/node-ca/0.log"
Jul 10 03:36:33.014592 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:33.014564    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:33.014731 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:33.014599    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:33.014731 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:33.014696    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 03:36:33.014731 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:33.014717    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:33.014846 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:33.014744    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm podName:a2b5f043-d820-4234-a832-021b63224ee9 nodeName:}" failed. No retries permitted until 2026-07-10 03:36:37.014730144 +0000 UTC m=+258.505725995 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnvwg" (UID: "a2b5f043-d820-4234-a832-021b63224ee9") : configmap "kube-rbac-proxy" not found
Jul 10 03:36:33.014846 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:33.014757    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls podName:a2b5f043-d820-4234-a832-021b63224ee9 nodeName:}" failed. No retries permitted until 2026-07-10 03:36:37.014751157 +0000 UTC m=+258.505747008 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnvwg" (UID: "a2b5f043-d820-4234-a832-021b63224ee9") : secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:33.793568 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:33.793528    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9" event={"ID":"a3c97055-ca89-454c-a482-d39a5b67060b","Type":"ContainerStarted","Data":"6d2fa39ecf004832c8aef764c5c7be0599799aeffb961c7d54b4d28458f4aae5"}
Jul 10 03:36:33.808070 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:33.808021    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-w4vh9" podStartSLOduration=2.059354758 podStartE2EDuration="3.80800641s" totalImagesPullingTime="1.748651652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:36:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:36:33.806863845 +0000 UTC m=+255.297859720" watchObservedRunningTime="2026-07-10 03:36:33.80800641 +0000 UTC m=+255.299002282"
Jul 10 03:36:33.880336 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:33.880263    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m9dtr_39488c5e-12b6-4a76-9383-28aa9c74d14c/serve-healthcheck-canary/0.log"
Jul 10 03:36:33.985125 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:33.985083    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:34.085562 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:34.085515    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:34.192029 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:34.191988    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:34.281242 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:34.281213    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-xtcdb_fbb60d0f-3458-4983-baa7-905512ac72d8/migrator/0.log"
Jul 10 03:36:34.380379 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:34.380290    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:34.486155 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:34.484047    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-xtcdb_fbb60d0f-3458-4983-baa7-905512ac72d8/graceful-termination/0.log"
Jul 10 03:36:34.486155 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:34.484186    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:34.539705 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:34.539672    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:34.676761 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:34.676679    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:34.680146 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:34.680124    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-j6bzq_6c312753-450d-4a92-822e-4638a7eafc6f/kube-storage-version-migrator-operator/0.log"
Jul 10 03:36:34.722213 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:34.722181    2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:34.722357 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:34.722231    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:34.722560 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:34.722547    2561 scope.go:117] "RemoveContainer" containerID="b7835bfe54622b89302a08d8317cbf2f9eb70ead481f6aa42265a982d51dabc8"
Jul 10 03:36:34.722737 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:34.722719    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-dcjt9_openshift-console-operator(b41fb4a3-4b8d-4c2d-b751-2283a10b4234)\"" pod="openshift-console-operator/console-operator-7866497494-dcjt9" podUID="b41fb4a3-4b8d-4c2d-b751-2283a10b4234"
Jul 10 03:36:35.882510 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:35.882478    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:36.080399 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:36.080346    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:36.283396 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:36.283289    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:36.477314 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:36.477278    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:36.680691 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:36.680638    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:37.049488 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:37.049404    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:37.049488 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:37.049444    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:37.049928 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:37.049551    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 03:36:37.049928 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:37.049578    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:37.049928 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:37.049611    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm podName:a2b5f043-d820-4234-a832-021b63224ee9 nodeName:}" failed. No retries permitted until 2026-07-10 03:36:45.049597433 +0000 UTC m=+266.540593284 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnvwg" (UID: "a2b5f043-d820-4234-a832-021b63224ee9") : configmap "kube-rbac-proxy" not found
Jul 10 03:36:37.049928 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:37.049675    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls podName:a2b5f043-d820-4234-a832-021b63224ee9 nodeName:}" failed. No retries permitted until 2026-07-10 03:36:45.049631512 +0000 UTC m=+266.540627363 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnvwg" (UID: "a2b5f043-d820-4234-a832-021b63224ee9") : secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:37.743843 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:37.743808    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:37.883878 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:37.883843    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:37.961359 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:37.961320    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:38.081379 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:38.081322    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:38.280575 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:38.280534    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:38.363038 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:38.362952    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:38.493711 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:38.493640    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:38.677153 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:38.677063    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:39.679900 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:39.679863    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:39.883599 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:39.883565    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:40.078337 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:40.078312    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:40.470302 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:40.470221    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:40.551902 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:40.551832    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:41.076587 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:41.076552    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:41.480469 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:41.480361    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:41.882410 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:41.882374    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:42.079086 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:42.079055    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:42.180898 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:42.180813    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:42.481406 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:42.481302    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:43.095944 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:43.095916    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:43.486135 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:43.486054    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:43.879891 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:43.879859    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:44.282172 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:44.282086    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:44.483827 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:44.483785    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:44.680247 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:44.680215    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:45.120345 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:45.120310    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:45.120514 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:45.120357    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:45.120895 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:45.120873    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2b5f043-d820-4234-a832-021b63224ee9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:45.122658 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:45.122628    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2b5f043-d820-4234-a832-021b63224ee9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnvwg\" (UID: \"a2b5f043-d820-4234-a832-021b63224ee9\") " pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:45.208052 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:45.208018    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tnvwg"
Jul 10 03:36:45.331985 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:45.331954    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tnvwg"]
Jul 10 03:36:45.335449 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:36:45.335424    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2b5f043_d820_4234_a832_021b63224ee9.slice/crio-69b5516f6bd2b87a94cf99bd3e91c95078fe0e8635b9f7ea27b5767e3e8c72b9 WatchSource:0}: Error finding container 69b5516f6bd2b87a94cf99bd3e91c95078fe0e8635b9f7ea27b5767e3e8c72b9: Status 404 returned error can't find the container with id 69b5516f6bd2b87a94cf99bd3e91c95078fe0e8635b9f7ea27b5767e3e8c72b9
Jul 10 03:36:45.834493 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:45.834457    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tnvwg" event={"ID":"a2b5f043-d820-4234-a832-021b63224ee9","Type":"ContainerStarted","Data":"69b5516f6bd2b87a94cf99bd3e91c95078fe0e8635b9f7ea27b5767e3e8c72b9"}
Jul 10 03:36:47.988483 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:47.988455    2561 scope.go:117] "RemoveContainer" containerID="b7835bfe54622b89302a08d8317cbf2f9eb70ead481f6aa42265a982d51dabc8"
Jul 10 03:36:49.554055 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:49.554023    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:49.653443 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:49.653419    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:49.706076 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:49.706045    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:49.850429 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:49.850394    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tnvwg" event={"ID":"a2b5f043-d820-4234-a832-021b63224ee9","Type":"ContainerStarted","Data":"a2981c57a1f652bc5cdfa85ff8c8567797f537699074d64c51ff913d3918a074"}
Jul 10 03:36:49.850587 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:49.850456    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tnvwg" event={"ID":"a2b5f043-d820-4234-a832-021b63224ee9","Type":"ContainerStarted","Data":"4079365cfbbd566ee541432ee03c6cc483bcd5098453ef033d75a63fd2afe189"}
Jul 10 03:36:49.852072 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:49.852047    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:36:49.852205 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:49.852117    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-dcjt9" event={"ID":"b41fb4a3-4b8d-4c2d-b751-2283a10b4234","Type":"ContainerStarted","Data":"3b48e077c71298012d534e3897154e8b2c0f3266467ec3b4d83022b3c8ffea1a"}
Jul 10 03:36:49.852400 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:49.852377    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:49.856863 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:49.856812    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:49.857146 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:49.857132    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-dcjt9"
Jul 10 03:36:49.868873 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:49.868831    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-dcjt9" podStartSLOduration=22.945638265 podStartE2EDuration="25.868818752s" totalImagesPullingTime="2.923180487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:36:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:36:49.867932159 +0000 UTC m=+271.358928051" watchObservedRunningTime="2026-07-10 03:36:49.868818752 +0000 UTC m=+271.359814710"
Jul 10 03:36:50.239035 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.238930    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:50.296994 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.296961    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:50.425755 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.425723    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-l8l4q"]
Jul 10 03:36:50.429074 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.429048    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-l8l4q"
Jul 10 03:36:50.431461 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.431202    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 10 03:36:50.431633 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.431534    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-r8qgb\""
Jul 10 03:36:50.431810 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.431776    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 10 03:36:50.439972 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.439948    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-l8l4q"]
Jul 10 03:36:50.508691 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.508594    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:50.568863 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.568828    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv42q\" (UniqueName: \"kubernetes.io/projected/9900998f-2e50-4777-867d-137431c31b88-kube-api-access-hv42q\") pod \"downloads-68c9cbb497-l8l4q\" (UID: \"9900998f-2e50-4777-867d-137431c31b88\") " pod="openshift-console/downloads-68c9cbb497-l8l4q"
Jul 10 03:36:50.575051 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.575027    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:50.669358 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.669319    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv42q\" (UniqueName: \"kubernetes.io/projected/9900998f-2e50-4777-867d-137431c31b88-kube-api-access-hv42q\") pod \"downloads-68c9cbb497-l8l4q\" (UID: \"9900998f-2e50-4777-867d-137431c31b88\") " pod="openshift-console/downloads-68c9cbb497-l8l4q"
Jul 10 03:36:50.676367 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.676333    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:50.678044 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.678023    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv42q\" (UniqueName: \"kubernetes.io/projected/9900998f-2e50-4777-867d-137431c31b88-kube-api-access-hv42q\") pod \"downloads-68c9cbb497-l8l4q\" (UID: \"9900998f-2e50-4777-867d-137431c31b88\") " pod="openshift-console/downloads-68c9cbb497-l8l4q"
Jul 10 03:36:50.741760 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.741730    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-l8l4q"
Jul 10 03:36:50.905157 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.905128    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-l8l4q"]
Jul 10 03:36:50.908232 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:36:50.908206    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9900998f_2e50_4777_867d_137431c31b88.slice/crio-37ca13b51f643e5a48f61aaa33871a22ef15ce2354b71018d74ae435b2c3f0cc WatchSource:0}: Error finding container 37ca13b51f643e5a48f61aaa33871a22ef15ce2354b71018d74ae435b2c3f0cc: Status 404 returned error can't find the container with id 37ca13b51f643e5a48f61aaa33871a22ef15ce2354b71018d74ae435b2c3f0cc
Jul 10 03:36:50.913780 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.913751    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:50.969621 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:50.969590    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:51.116865 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:51.116831    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:51.256025 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:51.255989    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:51.436953 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:51.436871    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:51.861796 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:51.861759    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-l8l4q" event={"ID":"9900998f-2e50-4777-867d-137431c31b88","Type":"ContainerStarted","Data":"37ca13b51f643e5a48f61aaa33871a22ef15ce2354b71018d74ae435b2c3f0cc"}
Jul 10 03:36:52.055575 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:52.055533    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:52.867758 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:52.867725    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tnvwg" event={"ID":"a2b5f043-d820-4234-a832-021b63224ee9","Type":"ContainerStarted","Data":"ded978005cdfd70b428b737286bad765e939f618981374f940c07fa25e38320e"}
Jul 10 03:36:52.885786 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:52.885737    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tnvwg" podStartSLOduration=17.202462428 podStartE2EDuration="23.885717236s" totalImagesPullingTime="6.683254808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:36:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:36:52.884177513 +0000 UTC m=+274.375173403" watchObservedRunningTime="2026-07-10 03:36:52.885717236 +0000 UTC m=+274.376713112"
Jul 10 03:36:54.095387 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.095351    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:54.149797 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.149766    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:54.474763 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.474675    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6ff58cd966-zl8zq"]
Jul 10 03:36:54.477838 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.477817    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.481277 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.481243    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 10 03:36:54.481277 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.481259    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j6wc5\""
Jul 10 03:36:54.481440 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.481334    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 10 03:36:54.481440 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.481420    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 10 03:36:54.481533 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.481482    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 10 03:36:54.481533 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.481495    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 10 03:36:54.488410 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.488373    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ff58cd966-zl8zq"]
Jul 10 03:36:54.494225 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.494204    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6abe9fca-96f7-4264-a59f-b098d2d94486-console-serving-cert\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.494338 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.494230    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6abe9fca-96f7-4264-a59f-b098d2d94486-console-oauth-config\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.494338 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.494249    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-oauth-serving-cert\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.494452 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.494394    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-console-config\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.494452 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.494426    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-service-ca\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.494556 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.494456    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkj2c\" (UniqueName: \"kubernetes.io/projected/6abe9fca-96f7-4264-a59f-b098d2d94486-kube-api-access-qkj2c\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.594972 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.594932    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-console-config\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.595168 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.594984    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-service-ca\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.595168 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.595011    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkj2c\" (UniqueName: \"kubernetes.io/projected/6abe9fca-96f7-4264-a59f-b098d2d94486-kube-api-access-qkj2c\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.595168 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.595041    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6abe9fca-96f7-4264-a59f-b098d2d94486-console-serving-cert\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.595168 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.595065    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6abe9fca-96f7-4264-a59f-b098d2d94486-console-oauth-config\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.595168 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.595086    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-oauth-serving-cert\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.595836 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.595770    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-console-config\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.595836 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.595770    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-service-ca\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.596024 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.595877    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-oauth-serving-cert\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.597966 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.597938    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6abe9fca-96f7-4264-a59f-b098d2d94486-console-serving-cert\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.598178 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.598158    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6abe9fca-96f7-4264-a59f-b098d2d94486-console-oauth-config\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.602780 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.602757    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkj2c\" (UniqueName: \"kubernetes.io/projected/6abe9fca-96f7-4264-a59f-b098d2d94486-kube-api-access-qkj2c\") pod \"console-6ff58cd966-zl8zq\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") " pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.722334 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.722297    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:54.774710 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.774622    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:54.788626 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.788601    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:36:54.923625 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:54.923592    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ff58cd966-zl8zq"]
Jul 10 03:36:54.927403 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:36:54.927365    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6abe9fca_96f7_4264_a59f_b098d2d94486.slice/crio-582f3a32dc18b1f84b5b2b433024f90eec5b76240e89266eb667464bde32f63e WatchSource:0}: Error finding container 582f3a32dc18b1f84b5b2b433024f90eec5b76240e89266eb667464bde32f63e: Status 404 returned error can't find the container with id 582f3a32dc18b1f84b5b2b433024f90eec5b76240e89266eb667464bde32f63e
Jul 10 03:36:55.881208 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:55.881160    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff58cd966-zl8zq" event={"ID":"6abe9fca-96f7-4264-a59f-b098d2d94486","Type":"ContainerStarted","Data":"582f3a32dc18b1f84b5b2b433024f90eec5b76240e89266eb667464bde32f63e"}
Jul 10 03:36:58.635948 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:58.635849    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:58.735878 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:58.735845    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:58.810545 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:58.810511    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:58.896766 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:58.896681    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff58cd966-zl8zq" event={"ID":"6abe9fca-96f7-4264-a59f-b098d2d94486","Type":"ContainerStarted","Data":"c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa"}
Jul 10 03:36:58.915575 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:58.915517    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6ff58cd966-zl8zq" podStartSLOduration=1.458824469 podStartE2EDuration="4.915497488s" totalImagesPullingTime="3.456673019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:36:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:36:58.914336644 +0000 UTC m=+280.405332518" watchObservedRunningTime="2026-07-10 03:36:58.915497488 +0000 UTC m=+280.406493362"
Jul 10 03:36:59.233640 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:59.233525    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:36:59.520266 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:59.520183    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5" podUID="ed31fb70-7e2b-4d84-946b-b00431c7e731"
Jul 10 03:36:59.520266 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:59.520183    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4rqhk" podUID="0843417a-dbe0-45e1-96c6-4510eb4d74ef"
Jul 10 03:36:59.520266 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:36:59.520202    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6" podUID="32888fa2-39e4-4de1-9a19-dee518a05139"
Jul 10 03:36:59.900634 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:59.900602    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:36:59.901110 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:59.900605    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:36:59.901110 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:36:59.900606    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:37:00.116836 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:00.116803    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:00.173885 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:00.173795    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:00.668797 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:00.668764    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:02.263541 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.263495    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:02.314257 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.314222    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:02.824417 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.824385    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:02.856683 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.856634    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-bmjvn"]
Jul 10 03:37:02.886502 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.886468    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-bmjvn"]
Jul 10 03:37:02.886692 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.886569    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:02.889715 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.889690    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 10 03:37:02.889715 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.889708    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 10 03:37:02.889908 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.889843    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 03:37:02.890735 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.890714    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 03:37:02.890891 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.890761    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-x8b5l\""
Jul 10 03:37:02.890891 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.890770    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 03:37:02.906214 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.906183    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:02.972990 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.972947    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ee46ee9f-caae-4525-a907-ccc8065de209-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-bmjvn\" (UID: \"ee46ee9f-caae-4525-a907-ccc8065de209\") " pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:02.972990 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.972994    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clm9z\" (UniqueName: \"kubernetes.io/projected/ee46ee9f-caae-4525-a907-ccc8065de209-kube-api-access-clm9z\") pod \"prometheus-operator-64785b6675-bmjvn\" (UID: \"ee46ee9f-caae-4525-a907-ccc8065de209\") " pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:02.973201 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.973033    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee46ee9f-caae-4525-a907-ccc8065de209-metrics-client-ca\") pod \"prometheus-operator-64785b6675-bmjvn\" (UID: \"ee46ee9f-caae-4525-a907-ccc8065de209\") " pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:02.973201 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:02.973065    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee46ee9f-caae-4525-a907-ccc8065de209-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-bmjvn\" (UID: \"ee46ee9f-caae-4525-a907-ccc8065de209\") " pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:03.074400 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.074363    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ee46ee9f-caae-4525-a907-ccc8065de209-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-bmjvn\" (UID: \"ee46ee9f-caae-4525-a907-ccc8065de209\") " pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:03.074400 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.074403    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clm9z\" (UniqueName: \"kubernetes.io/projected/ee46ee9f-caae-4525-a907-ccc8065de209-kube-api-access-clm9z\") pod \"prometheus-operator-64785b6675-bmjvn\" (UID: \"ee46ee9f-caae-4525-a907-ccc8065de209\") " pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:03.074682 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.074427    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee46ee9f-caae-4525-a907-ccc8065de209-metrics-client-ca\") pod \"prometheus-operator-64785b6675-bmjvn\" (UID: \"ee46ee9f-caae-4525-a907-ccc8065de209\") " pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:03.074682 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.074458    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee46ee9f-caae-4525-a907-ccc8065de209-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-bmjvn\" (UID: \"ee46ee9f-caae-4525-a907-ccc8065de209\") " pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:03.075244 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.075219    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee46ee9f-caae-4525-a907-ccc8065de209-metrics-client-ca\") pod \"prometheus-operator-64785b6675-bmjvn\" (UID: \"ee46ee9f-caae-4525-a907-ccc8065de209\") " pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:03.077157 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.077131    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee46ee9f-caae-4525-a907-ccc8065de209-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-bmjvn\" (UID: \"ee46ee9f-caae-4525-a907-ccc8065de209\") " pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:03.077256 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.077143    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ee46ee9f-caae-4525-a907-ccc8065de209-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-bmjvn\" (UID: \"ee46ee9f-caae-4525-a907-ccc8065de209\") " pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:03.082043 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.082022    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clm9z\" (UniqueName: \"kubernetes.io/projected/ee46ee9f-caae-4525-a907-ccc8065de209-kube-api-access-clm9z\") pod \"prometheus-operator-64785b6675-bmjvn\" (UID: \"ee46ee9f-caae-4525-a907-ccc8065de209\") " pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:03.084497 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.084471    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:03.198191 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.198161    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn"
Jul 10 03:37:03.377078 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.376992    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:37:03.377078 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.377031    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:37:03.377591 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.377123    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:37:03.379852 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.379812    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0843417a-dbe0-45e1-96c6-4510eb4d74ef-metrics-tls\") pod \"dns-default-4rqhk\" (UID: \"0843417a-dbe0-45e1-96c6-4510eb4d74ef\") " pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:37:03.379964 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.379928    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed31fb70-7e2b-4d84-946b-b00431c7e731-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9gjc5\" (UID: \"ed31fb70-7e2b-4d84-946b-b00431c7e731\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:37:03.380029 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.379994    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"image-registry-7f84d8746d-csqp6\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") " pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:37:03.505768 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.505730    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tqxc2\""
Jul 10 03:37:03.505768 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.505764    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dk2pk\""
Jul 10 03:37:03.506023 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.505736    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2bqsd\""
Jul 10 03:37:03.512159 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.512134    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"
Jul 10 03:37:03.512306 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.512247    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:37:03.512374 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:03.512322    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:37:04.789681 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:04.789626    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:37:04.790094 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:04.789749    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:37:04.791342 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:04.791316    2561 patch_prober.go:28] interesting pod/console-6ff58cd966-zl8zq container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused" start-of-body=
Jul 10 03:37:04.791455 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:04.791362    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6ff58cd966-zl8zq" podUID="6abe9fca-96f7-4264-a59f-b098d2d94486" containerName="console" probeResult="failure" output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused"
Jul 10 03:37:06.670493 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:06.670451    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:06.736275 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:06.736229    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:08.315740 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.315712    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-9gjc5"]
Jul 10 03:37:08.318644 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:37:08.318612    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded31fb70_7e2b_4d84_946b_b00431c7e731.slice/crio-1ae2c42760264ad363d23fb1f0bcabb05ff8bbe3b38014431df9e88dde1c9c54 WatchSource:0}: Error finding container 1ae2c42760264ad363d23fb1f0bcabb05ff8bbe3b38014431df9e88dde1c9c54: Status 404 returned error can't find the container with id 1ae2c42760264ad363d23fb1f0bcabb05ff8bbe3b38014431df9e88dde1c9c54
Jul 10 03:37:08.366332 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.366310    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f84d8746d-csqp6"]
Jul 10 03:37:08.368605 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:37:08.368578    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32888fa2_39e4_4de1_9a19_dee518a05139.slice/crio-cb90d09dc96356d6b638faacfff79499e4d9fb7cd599730e86c5870b2d57e740 WatchSource:0}: Error finding container cb90d09dc96356d6b638faacfff79499e4d9fb7cd599730e86c5870b2d57e740: Status 404 returned error can't find the container with id cb90d09dc96356d6b638faacfff79499e4d9fb7cd599730e86c5870b2d57e740
Jul 10 03:37:08.540453 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.540303    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4rqhk"]
Jul 10 03:37:08.543062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.542995    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-bmjvn"]
Jul 10 03:37:08.544218 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:37:08.544188    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0843417a_dbe0_45e1_96c6_4510eb4d74ef.slice/crio-9b3f5fd8c8cfee401b5986fd5587b50307b8caed38dd069f31661ed53a8fb613 WatchSource:0}: Error finding container 9b3f5fd8c8cfee401b5986fd5587b50307b8caed38dd069f31661ed53a8fb613: Status 404 returned error can't find the container with id 9b3f5fd8c8cfee401b5986fd5587b50307b8caed38dd069f31661ed53a8fb613
Jul 10 03:37:08.548110 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:37:08.548084    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee46ee9f_caae_4525_a907_ccc8065de209.slice/crio-074ab535c6516d00d223356ca28ed94014ebad490874e6c4a4148d75ba10e74b WatchSource:0}: Error finding container 074ab535c6516d00d223356ca28ed94014ebad490874e6c4a4148d75ba10e74b: Status 404 returned error can't find the container with id 074ab535c6516d00d223356ca28ed94014ebad490874e6c4a4148d75ba10e74b
Jul 10 03:37:08.937898 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.936809    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-l8l4q" event={"ID":"9900998f-2e50-4777-867d-137431c31b88","Type":"ContainerStarted","Data":"767499e5a9777f6facd6d45d29fd4247f148e1618450ed1bd421d83cc92e15ad"}
Jul 10 03:37:08.937898 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.937839    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-l8l4q"
Jul 10 03:37:08.939876 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.939801    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5" event={"ID":"ed31fb70-7e2b-4d84-946b-b00431c7e731","Type":"ContainerStarted","Data":"1ae2c42760264ad363d23fb1f0bcabb05ff8bbe3b38014431df9e88dde1c9c54"}
Jul 10 03:37:08.942547 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.942260    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-l8l4q"
Jul 10 03:37:08.942547 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.942463    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4rqhk" event={"ID":"0843417a-dbe0-45e1-96c6-4510eb4d74ef","Type":"ContainerStarted","Data":"9b3f5fd8c8cfee401b5986fd5587b50307b8caed38dd069f31661ed53a8fb613"}
Jul 10 03:37:08.948594 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.947907    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6" event={"ID":"32888fa2-39e4-4de1-9a19-dee518a05139","Type":"ContainerStarted","Data":"31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e"}
Jul 10 03:37:08.948594 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.947956    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6" event={"ID":"32888fa2-39e4-4de1-9a19-dee518a05139","Type":"ContainerStarted","Data":"cb90d09dc96356d6b638faacfff79499e4d9fb7cd599730e86c5870b2d57e740"}
Jul 10 03:37:08.948594 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.948531    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:37:08.953046 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.953016    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn" event={"ID":"ee46ee9f-caae-4525-a907-ccc8065de209","Type":"ContainerStarted","Data":"074ab535c6516d00d223356ca28ed94014ebad490874e6c4a4148d75ba10e74b"}
Jul 10 03:37:08.955598 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.955444    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-l8l4q" podStartSLOduration=1.5928171020000002 podStartE2EDuration="18.955424433s" totalImagesPullingTime="17.362607331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:08.954353305 +0000 UTC m=+290.445349182" watchObservedRunningTime="2026-07-10 03:37:08.955424433 +0000 UTC m=+290.446420310"
Jul 10 03:37:08.975110 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:08.974825    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6" podStartSLOduration=289.974810254 podStartE2EDuration="4m49.974810254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:08.974110732 +0000 UTC m=+290.465106607" watchObservedRunningTime="2026-07-10 03:37:08.974810254 +0000 UTC m=+290.465806130"
Jul 10 03:37:09.101118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:09.101032    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:09.188640 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:09.188526    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:11.109973 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:11.109925    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:11.163848 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:11.163795    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:11.768724 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:11.768684    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:11.968767 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:11.968682    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5" event={"ID":"ed31fb70-7e2b-4d84-946b-b00431c7e731","Type":"ContainerStarted","Data":"5659d4641afaaf7a3a55b72afe6ae80c501704664b4955711c3c6b700b982a60"}
Jul 10 03:37:11.970859 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:11.970830    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4rqhk" event={"ID":"0843417a-dbe0-45e1-96c6-4510eb4d74ef","Type":"ContainerStarted","Data":"3f8295bdad5ee43f8aaf49bc1f9384a3d3e428563da3053528bedcd68fc98393"}
Jul 10 03:37:11.971002 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:11.970879    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4rqhk" event={"ID":"0843417a-dbe0-45e1-96c6-4510eb4d74ef","Type":"ContainerStarted","Data":"de8315c62ab9b01fec7ff24f1c226a1bda128dfdccc420aa0b5bff944714b7dc"}
Jul 10 03:37:11.971002 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:11.970906    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:37:11.972920 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:11.972884    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn" event={"ID":"ee46ee9f-caae-4525-a907-ccc8065de209","Type":"ContainerStarted","Data":"af062f6ce0e496d4e3092830b1e3bd259d4589bad687a7d5b2901b53b7c380e1"}
Jul 10 03:37:11.973027 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:11.972928    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn" event={"ID":"ee46ee9f-caae-4525-a907-ccc8065de209","Type":"ContainerStarted","Data":"c2ad4c9d79d54f1747b0fd12b01a50c26987a86926b85987e97f3383ecf08c56"}
Jul 10 03:37:11.985686 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:11.985618    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-9gjc5" podStartSLOduration=284.889796612 podStartE2EDuration="4m47.985605309s" totalImagesPullingTime="3.095808697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:11.985061729 +0000 UTC m=+293.476057601" watchObservedRunningTime="2026-07-10 03:37:11.985605309 +0000 UTC m=+293.476601182"
Jul 10 03:37:12.001583 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:12.001521    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4rqhk" podStartSLOduration=256.125304788 podStartE2EDuration="4m19.001502871s" totalImagesPullingTime="2.876198083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:12.000265083 +0000 UTC m=+293.491260976" watchObservedRunningTime="2026-07-10 03:37:12.001502871 +0000 UTC m=+293.492498746"
Jul 10 03:37:12.016935 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:12.016869    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-bmjvn" podStartSLOduration=7.149903845 podStartE2EDuration="10.016846994s" totalImagesPullingTime="2.866943149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:37:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:12.015774123 +0000 UTC m=+293.506770018" watchObservedRunningTime="2026-07-10 03:37:12.016846994 +0000 UTC m=+293.507842867"
Jul 10 03:37:12.096210 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:12.096174    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:12.153779 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:12.153745    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:13.948182 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:13.948124    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:13.989237 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:37:13.989193    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-p7l8g" podUID="776389fa-994a-4f3b-bcfb-48c486232e9b"
Jul 10 03:37:14.181395 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.181360    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:14.209391 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.209238    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-v49v2"]
Jul 10 03:37:14.257829 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.257784    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:14.282513 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.282492    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.286773 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.286753    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 03:37:14.286962 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.286944    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 03:37:14.287215 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.287197    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 03:37:14.287307 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.286890    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8h9sp\""
Jul 10 03:37:14.319319 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.319281    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:14.375762 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.375721    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81c9f818-0f55-425e-a28e-a921feecbdf5-metrics-client-ca\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.375959 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.375780    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfm98\" (UniqueName: \"kubernetes.io/projected/81c9f818-0f55-425e-a28e-a921feecbdf5-kube-api-access-bfm98\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.375959 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.375830    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/81c9f818-0f55-425e-a28e-a921feecbdf5-root\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.375959 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.375867    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-accelerators-collector-config\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.375959 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.375894    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81c9f818-0f55-425e-a28e-a921feecbdf5-sys\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.375959 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.375944    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.376200 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.375972    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-tls\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.376200 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.376004    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-textfile\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.376200 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.376038    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-wtmp\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.421724 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.421689    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:14.477101 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.477014    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfm98\" (UniqueName: \"kubernetes.io/projected/81c9f818-0f55-425e-a28e-a921feecbdf5-kube-api-access-bfm98\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.477436 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.477413    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/81c9f818-0f55-425e-a28e-a921feecbdf5-root\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.477562 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.477339    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/81c9f818-0f55-425e-a28e-a921feecbdf5-root\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.477726 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.477699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-accelerators-collector-config\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.477843 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.477829    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81c9f818-0f55-425e-a28e-a921feecbdf5-sys\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.477977 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.477963    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.479376 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.479100    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-tls\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.479592 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.479565    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-textfile\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.479827 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.479811    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-wtmp\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.480341 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.480322    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81c9f818-0f55-425e-a28e-a921feecbdf5-metrics-client-ca\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.481737 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:37:14.479220    2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 10 03:37:14.481998 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.479000    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81c9f818-0f55-425e-a28e-a921feecbdf5-sys\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.482110 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.480710    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-wtmp\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.482203 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.481344    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81c9f818-0f55-425e-a28e-a921feecbdf5-metrics-client-ca\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.482302 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.481572    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.482926 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.482503    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-textfile\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.483060 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:37:14.482909    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-tls podName:81c9f818-0f55-425e-a28e-a921feecbdf5 nodeName:}" failed. No retries permitted until 2026-07-10 03:37:14.982439281 +0000 UTC m=+296.473435138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-tls") pod "node-exporter-v49v2" (UID: "81c9f818-0f55-425e-a28e-a921feecbdf5") : secret "node-exporter-tls" not found
Jul 10 03:37:14.483481 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.483466    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-accelerators-collector-config\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.486241 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.486211    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfm98\" (UniqueName: \"kubernetes.io/projected/81c9f818-0f55-425e-a28e-a921feecbdf5-kube-api-access-bfm98\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.519005 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.518963    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:14.618004 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.617964    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:14.682118 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.682074    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:14.783575 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.783470    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:14.789689 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.789633    2561 patch_prober.go:28] interesting pod/console-6ff58cd966-zl8zq container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused" start-of-body=
Jul 10 03:37:14.789844 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.789747    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6ff58cd966-zl8zq" podUID="6abe9fca-96f7-4264-a59f-b098d2d94486" containerName="console" probeResult="failure" output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused"
Jul 10 03:37:14.883638 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.883600    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:14.985301 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.985232    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-tls\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:14.988718 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:14.988531    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/81c9f818-0f55-425e-a28e-a921feecbdf5-node-exporter-tls\") pod \"node-exporter-v49v2\" (UID: \"81c9f818-0f55-425e-a28e-a921feecbdf5\") " pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:15.178253 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:15.178013    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:15.198291 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:15.198254    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v49v2"
Jul 10 03:37:15.259041 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:37:15.259007    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c9f818_0f55_425e_a28e_a921feecbdf5.slice/crio-c1310c1a19cf50104aee037dba1f49e42468ede967e62f92e12acfcfde38b995 WatchSource:0}: Error finding container c1310c1a19cf50104aee037dba1f49e42468ede967e62f92e12acfcfde38b995: Status 404 returned error can't find the container with id c1310c1a19cf50104aee037dba1f49e42468ede967e62f92e12acfcfde38b995
Jul 10 03:37:15.275239 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:15.275209    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:15.779728 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:15.779684    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:15.995031 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:15.994989    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v49v2" event={"ID":"81c9f818-0f55-425e-a28e-a921feecbdf5","Type":"ContainerStarted","Data":"c1310c1a19cf50104aee037dba1f49e42468ede967e62f92e12acfcfde38b995"}
Jul 10 03:37:16.197871 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.197833    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7cb5995774-8lqs7"]
Jul 10 03:37:16.232961 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.232917    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7cb5995774-8lqs7"]
Jul 10 03:37:16.233148 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.233042    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.237389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.237339    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 10 03:37:16.237389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.237386    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 10 03:37:16.237624 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.237613    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 10 03:37:16.238345 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.238323    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 10 03:37:16.238533 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.238480    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c3f6b2i7n7sit\""
Jul 10 03:37:16.238688 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.238637    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8l89f\""
Jul 10 03:37:16.239386 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.238866    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 10 03:37:16.296640 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.296603    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/293c39d2-6ccb-49b0-adee-2c3952399b07-metrics-client-ca\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.296830 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.296668    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.296830 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.296758    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-grpc-tls\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.296938 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.296848    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-tls\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.296938 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.296881    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.296938 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.296923    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv4s8\" (UniqueName: \"kubernetes.io/projected/293c39d2-6ccb-49b0-adee-2c3952399b07-kube-api-access-gv4s8\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.297048 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.296996    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.297048 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.297037    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.398369 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.398329    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.398553 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.398381    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.398553 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.398423    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/293c39d2-6ccb-49b0-adee-2c3952399b07-metrics-client-ca\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.398553 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.398450    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.398801 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.398560    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-grpc-tls\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.398801 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.398621    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-tls\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.398801 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.398676    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.398801 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.398713    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv4s8\" (UniqueName: \"kubernetes.io/projected/293c39d2-6ccb-49b0-adee-2c3952399b07-kube-api-access-gv4s8\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.399410 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.399347    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/293c39d2-6ccb-49b0-adee-2c3952399b07-metrics-client-ca\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.401841 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.401815    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-grpc-tls\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.402345 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.402318    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.402440 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.402423    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.402793 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.402769    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.403263 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.403244    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.403377 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.403348    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/293c39d2-6ccb-49b0-adee-2c3952399b07-secret-thanos-querier-tls\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.407089 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.407065    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv4s8\" (UniqueName: \"kubernetes.io/projected/293c39d2-6ccb-49b0-adee-2c3952399b07-kube-api-access-gv4s8\") pod \"thanos-querier-7cb5995774-8lqs7\" (UID: \"293c39d2-6ccb-49b0-adee-2c3952399b07\") " pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.548752 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.548643    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:16.904218 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:16.904185    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7cb5995774-8lqs7"]
Jul 10 03:37:16.907741 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:37:16.907704    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod293c39d2_6ccb_49b0_adee_2c3952399b07.slice/crio-8880a87707d45f033d234fcafdd6e62d2c5a1f1c8520af27e0bad7bf5cd1ee4b WatchSource:0}: Error finding container 8880a87707d45f033d234fcafdd6e62d2c5a1f1c8520af27e0bad7bf5cd1ee4b: Status 404 returned error can't find the container with id 8880a87707d45f033d234fcafdd6e62d2c5a1f1c8520af27e0bad7bf5cd1ee4b
Jul 10 03:37:17.008151 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:17.008104    2561 generic.go:356] "Generic (PLEG): container finished" podID="81c9f818-0f55-425e-a28e-a921feecbdf5" containerID="41521b7c2debb5cc91fa0f9fad3936d8ba5cd5a7ef02d1e93595bff8f64c6bda" exitCode=0
Jul 10 03:37:17.008606 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:17.008208    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v49v2" event={"ID":"81c9f818-0f55-425e-a28e-a921feecbdf5","Type":"ContainerDied","Data":"41521b7c2debb5cc91fa0f9fad3936d8ba5cd5a7ef02d1e93595bff8f64c6bda"}
Jul 10 03:37:17.009939 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:17.009912    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7" event={"ID":"293c39d2-6ccb-49b0-adee-2c3952399b07","Type":"ContainerStarted","Data":"8880a87707d45f033d234fcafdd6e62d2c5a1f1c8520af27e0bad7bf5cd1ee4b"}
Jul 10 03:37:17.174372 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:17.174337    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:17.274732 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:17.274447    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:17.436766 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:17.436722    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:17.635396 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:17.635358    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:18.016049 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.016010    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v49v2" event={"ID":"81c9f818-0f55-425e-a28e-a921feecbdf5","Type":"ContainerStarted","Data":"241226a44d289e69b643109f4a44f3a2de5b99d4857b0cdb148cd25304c0c9f1"}
Jul 10 03:37:18.016524 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.016073    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v49v2" event={"ID":"81c9f818-0f55-425e-a28e-a921feecbdf5","Type":"ContainerStarted","Data":"77eaa0e8a32e7d6bd79770ed3ad71439a8c87e4b5e526d3f006e6f6e6dcb1dac"}
Jul 10 03:37:18.038898 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.038850    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-v49v2" podStartSLOduration=2.4551841420000002 podStartE2EDuration="4.038832511s" totalImagesPullingTime="1.486723934s" totalInitContainerRuntime="96.924435ms" isStatefulPod=true podCreationTimestamp="2026-07-10 03:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:18.037213285 +0000 UTC m=+299.528209177" watchObservedRunningTime="2026-07-10 03:37:18.038832511 +0000 UTC m=+299.529828384"
Jul 10 03:37:18.079174 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.079132    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:18.520053 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.520019    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"]
Jul 10 03:37:18.546196 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.546160    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"]
Jul 10 03:37:18.546373 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.546306    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.549862 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.549835    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 10 03:37:18.550145 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.550123    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 10 03:37:18.550326 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.550171    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-kg98j\""
Jul 10 03:37:18.550326 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.550219    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 10 03:37:18.550438 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.550335    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 10 03:37:18.550438 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.550331    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-90ki608f9t2kn\""
Jul 10 03:37:18.620872 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.620840    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/018616ab-fbf6-4171-bf25-c39c8b8446d4-secret-metrics-server-tls\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.621069 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.620881    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46xh6\" (UniqueName: \"kubernetes.io/projected/018616ab-fbf6-4171-bf25-c39c8b8446d4-kube-api-access-46xh6\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.621069 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.620927    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/018616ab-fbf6-4171-bf25-c39c8b8446d4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.621069 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.620996    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/018616ab-fbf6-4171-bf25-c39c8b8446d4-metrics-server-audit-profiles\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.621226 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.621101    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/018616ab-fbf6-4171-bf25-c39c8b8446d4-audit-log\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.621226 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.621136    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/018616ab-fbf6-4171-bf25-c39c8b8446d4-client-ca-bundle\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.621226 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.621168    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/018616ab-fbf6-4171-bf25-c39c8b8446d4-secret-metrics-server-client-certs\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.722250 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.722192    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/018616ab-fbf6-4171-bf25-c39c8b8446d4-metrics-server-audit-profiles\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.722446 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.722370    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/018616ab-fbf6-4171-bf25-c39c8b8446d4-audit-log\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.722446 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.722408    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/018616ab-fbf6-4171-bf25-c39c8b8446d4-client-ca-bundle\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.722446 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.722435    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/018616ab-fbf6-4171-bf25-c39c8b8446d4-secret-metrics-server-client-certs\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.722609 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.722481    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/018616ab-fbf6-4171-bf25-c39c8b8446d4-secret-metrics-server-tls\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.722609 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.722505    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46xh6\" (UniqueName: \"kubernetes.io/projected/018616ab-fbf6-4171-bf25-c39c8b8446d4-kube-api-access-46xh6\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.722729 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.722676    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/018616ab-fbf6-4171-bf25-c39c8b8446d4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.723133 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.723085    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/018616ab-fbf6-4171-bf25-c39c8b8446d4-audit-log\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.723360 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.723340    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/018616ab-fbf6-4171-bf25-c39c8b8446d4-metrics-server-audit-profiles\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.723631 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.723603    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/018616ab-fbf6-4171-bf25-c39c8b8446d4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.725412 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.725391    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/018616ab-fbf6-4171-bf25-c39c8b8446d4-client-ca-bundle\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.725529 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.725508    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/018616ab-fbf6-4171-bf25-c39c8b8446d4-secret-metrics-server-client-certs\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.725635 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.725607    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/018616ab-fbf6-4171-bf25-c39c8b8446d4-secret-metrics-server-tls\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.732122 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.732085    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46xh6\" (UniqueName: \"kubernetes.io/projected/018616ab-fbf6-4171-bf25-c39c8b8446d4-kube-api-access-46xh6\") pod \"metrics-server-7b9d8fb85f-mpsmw\" (UID: \"018616ab-fbf6-4171-bf25-c39c8b8446d4\") " pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.857201 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.857162    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:18.945732 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.945604    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:37:18.945910 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.945897    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:37:18.961080 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:18.961055    2561 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 03:37:19.008208 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.008069    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.028254    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"]
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:37:19.031411    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018616ab_fbf6_4171_bf25_c39c8b8446d4.slice/crio-f82b2b240c2a2e4463d7af213ad8e47e5ee88362224adcdba5ecd35c4d04a2f6 WatchSource:0}: Error finding container f82b2b240c2a2e4463d7af213ad8e47e5ee88362224adcdba5ecd35c4d04a2f6: Status 404 returned error can't find the container with id f82b2b240c2a2e4463d7af213ad8e47e5ee88362224adcdba5ecd35c4d04a2f6
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.043567    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"]
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.043621    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"]
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.043691    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.045979    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xrqjp\""
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.046189    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.066409    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.126572    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a3cdab6f-eed4-4108-93fd-0231e7c3b4ff-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-7gmsg\" (UID: \"a3cdab6f-eed4-4108-93fd-0231e7c3b4ff\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.126733    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7gdn\" (UniqueName: \"kubernetes.io/projected/a3cdab6f-eed4-4108-93fd-0231e7c3b4ff-kube-api-access-f7gdn\") pod \"monitoring-plugin-5f68df7f58-7gmsg\" (UID: \"a3cdab6f-eed4-4108-93fd-0231e7c3b4ff\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.228007    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a3cdab6f-eed4-4108-93fd-0231e7c3b4ff-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-7gmsg\" (UID: \"a3cdab6f-eed4-4108-93fd-0231e7c3b4ff\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.228078    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7gdn\" (UniqueName: \"kubernetes.io/projected/a3cdab6f-eed4-4108-93fd-0231e7c3b4ff-kube-api-access-f7gdn\") pod \"monitoring-plugin-5f68df7f58-7gmsg\" (UID: \"a3cdab6f-eed4-4108-93fd-0231e7c3b4ff\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.228876    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.230522    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a3cdab6f-eed4-4108-93fd-0231e7c3b4ff-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-7gmsg\" (UID: \"a3cdab6f-eed4-4108-93fd-0231e7c3b4ff\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.236822    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7gdn\" (UniqueName: \"kubernetes.io/projected/a3cdab6f-eed4-4108-93fd-0231e7c3b4ff-kube-api-access-f7gdn\") pod \"monitoring-plugin-5f68df7f58-7gmsg\" (UID: \"a3cdab6f-eed4-4108-93fd-0231e7c3b4ff\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.356577    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:19.516035    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"]
Jul 10 03:37:19.893389 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:37:19.520362    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3cdab6f_eed4_4108_93fd_0231e7c3b4ff.slice/crio-c034269ed00633ea674ebefe805b64bfe057b3a9c43a7cba7d48be58b5acc380 WatchSource:0}: Error finding container c034269ed00633ea674ebefe805b64bfe057b3a9c43a7cba7d48be58b5acc380: Status 404 returned error can't find the container with id c034269ed00633ea674ebefe805b64bfe057b3a9c43a7cba7d48be58b5acc380
Jul 10 03:37:20.024815 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.024773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw" event={"ID":"018616ab-fbf6-4171-bf25-c39c8b8446d4","Type":"ContainerStarted","Data":"f82b2b240c2a2e4463d7af213ad8e47e5ee88362224adcdba5ecd35c4d04a2f6"}
Jul 10 03:37:20.027015 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.026986    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7" event={"ID":"293c39d2-6ccb-49b0-adee-2c3952399b07","Type":"ContainerStarted","Data":"37b7527256585dd282bd28028e69bd3553efdf9f7a510568a0897e7585e312ee"}
Jul 10 03:37:20.027159 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.027056    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7" event={"ID":"293c39d2-6ccb-49b0-adee-2c3952399b07","Type":"ContainerStarted","Data":"a86b36e9c17b969717805b710a7405ccbfa4de0fc5ea2d17dccbf424846ffaac"}
Jul 10 03:37:20.027159 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.027095    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7" event={"ID":"293c39d2-6ccb-49b0-adee-2c3952399b07","Type":"ContainerStarted","Data":"0ebb56b2b65c05aa6bdccac0a3c71a5c3c196b9d1e994f37cf4238a7b28c3952"}
Jul 10 03:37:20.028170 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.028145    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg" event={"ID":"a3cdab6f-eed4-4108-93fd-0231e7c3b4ff","Type":"ContainerStarted","Data":"c034269ed00633ea674ebefe805b64bfe057b3a9c43a7cba7d48be58b5acc380"}
Jul 10 03:37:20.281738 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.281641    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ff58cd966-zl8zq"]
Jul 10 03:37:20.306543 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.306508    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dc757fcd9-xn7db"]
Jul 10 03:37:20.328871 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.328837    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dc757fcd9-xn7db"]
Jul 10 03:37:20.329047 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.328965    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.338543 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.338461    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 10 03:37:20.439500 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.439463    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpvw6\" (UniqueName: \"kubernetes.io/projected/4e973bd5-3603-4502-86bc-097e3cc89bca-kube-api-access-mpvw6\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.439713 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.439556    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-console-config\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.439713 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.439610    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e973bd5-3603-4502-86bc-097e3cc89bca-console-serving-cert\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.439713 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.439635    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-service-ca\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.439906 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.439777    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e973bd5-3603-4502-86bc-097e3cc89bca-console-oauth-config\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.439906 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.439815    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-oauth-serving-cert\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.440013 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.439962    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-trusted-ca-bundle\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.493048 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.493012    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:20.542086 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.541298    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-console-config\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.542086 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.541374    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e973bd5-3603-4502-86bc-097e3cc89bca-console-serving-cert\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.542086 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.541400    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-service-ca\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.542086 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.541435    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e973bd5-3603-4502-86bc-097e3cc89bca-console-oauth-config\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.542086 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.541471    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-oauth-serving-cert\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.542086 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.541522    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-trusted-ca-bundle\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.542086 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.541556    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpvw6\" (UniqueName: \"kubernetes.io/projected/4e973bd5-3603-4502-86bc-097e3cc89bca-kube-api-access-mpvw6\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.542697 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.542673    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-console-config\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.543714 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.543686    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-service-ca\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.543714 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.543694    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-oauth-serving-cert\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.544335 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.544305    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-trusted-ca-bundle\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.546210 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.546116    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e973bd5-3603-4502-86bc-097e3cc89bca-console-oauth-config\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.546327 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.546295    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e973bd5-3603-4502-86bc-097e3cc89bca-console-serving-cert\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.550023 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.549979    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpvw6\" (UniqueName: \"kubernetes.io/projected/4e973bd5-3603-4502-86bc-097e3cc89bca-kube-api-access-mpvw6\") pod \"console-7dc757fcd9-xn7db\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") " pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:20.644279 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:20.644239    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:21.120209 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:21.120163    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:21.924675 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:21.924603    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dc757fcd9-xn7db"]
Jul 10 03:37:21.929694 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:37:21.929644    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e973bd5_3603_4502_86bc_097e3cc89bca.slice/crio-714d30b49dd1595930891c9d4d979f04dcd0d607c26a93f3b69ad09f67a77394 WatchSource:0}: Error finding container 714d30b49dd1595930891c9d4d979f04dcd0d607c26a93f3b69ad09f67a77394: Status 404 returned error can't find the container with id 714d30b49dd1595930891c9d4d979f04dcd0d607c26a93f3b69ad09f67a77394
Jul 10 03:37:21.982795 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:21.982767    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4rqhk"
Jul 10 03:37:22.041194 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:22.041104    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg" event={"ID":"a3cdab6f-eed4-4108-93fd-0231e7c3b4ff","Type":"ContainerStarted","Data":"75f53f8c506fb46bbae321f6fd1a4b703758d86de620eadbe65680d02b81d31a"}
Jul 10 03:37:22.041194 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:22.041174    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"
Jul 10 03:37:22.043124 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:22.043099    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc757fcd9-xn7db" event={"ID":"4e973bd5-3603-4502-86bc-097e3cc89bca","Type":"ContainerStarted","Data":"87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41"}
Jul 10 03:37:22.043512 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:22.043493    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc757fcd9-xn7db" event={"ID":"4e973bd5-3603-4502-86bc-097e3cc89bca","Type":"ContainerStarted","Data":"714d30b49dd1595930891c9d4d979f04dcd0d607c26a93f3b69ad09f67a77394"}
Jul 10 03:37:22.045404 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:22.045382    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw" event={"ID":"018616ab-fbf6-4171-bf25-c39c8b8446d4","Type":"ContainerStarted","Data":"814bedef680f1eb06a3d9bb94408eb249391eb2984a9d36a6e0474e3120c3392"}
Jul 10 03:37:22.047545 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:22.047526    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg"
Jul 10 03:37:22.074203 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:22.074145    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7gmsg" podStartSLOduration=0.806685939 podStartE2EDuration="3.074124258s" totalImagesPullingTime="2.267438319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:37:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:22.072390959 +0000 UTC m=+303.563386831" watchObservedRunningTime="2026-07-10 03:37:22.074124258 +0000 UTC m=+303.565120132"
Jul 10 03:37:22.143007 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:22.142911    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw" podStartSLOduration=1.542196621 podStartE2EDuration="4.142896543s" totalImagesPullingTime="2.600699922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:37:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:22.142475431 +0000 UTC m=+303.633471305" watchObservedRunningTime="2026-07-10 03:37:22.142896543 +0000 UTC m=+303.633892395"
Jul 10 03:37:22.143730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:22.143698    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dc757fcd9-xn7db" podStartSLOduration=2.143687456 podStartE2EDuration="2.143687456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:37:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:22.117589843 +0000 UTC m=+303.608585716" watchObservedRunningTime="2026-07-10 03:37:22.143687456 +0000 UTC m=+303.634683330"
Jul 10 03:37:22.487200 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:22.487167    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:23.034784 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:23.034752    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f84d8746d-csqp6"]
Jul 10 03:37:23.039510 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:23.039486    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:37:23.052862 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:23.052829    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7" event={"ID":"293c39d2-6ccb-49b0-adee-2c3952399b07","Type":"ContainerStarted","Data":"a82723f97e63efcd02e131fd83d3d056584da81736a97a469451dfe94a9dd1a7"}
Jul 10 03:37:23.052999 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:23.052902    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7" event={"ID":"293c39d2-6ccb-49b0-adee-2c3952399b07","Type":"ContainerStarted","Data":"a60c2b7887ec0621ffe8167e555f4d9f2e3b0f4339e722ac647b0da561f21947"}
Jul 10 03:37:23.052999 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:23.052943    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7" event={"ID":"293c39d2-6ccb-49b0-adee-2c3952399b07","Type":"ContainerStarted","Data":"044cf16b19e62884ec504dafe68d2f42c23c0cce047f551490608cfc35a46dc3"}
Jul 10 03:37:23.102543 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:23.102487    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7" podStartSLOduration=2.175768569 podStartE2EDuration="7.102472506s" totalImagesPullingTime="4.926703937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:37:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:23.10097407 +0000 UTC m=+304.591969942" watchObservedRunningTime="2026-07-10 03:37:23.102472506 +0000 UTC m=+304.593468378"
Jul 10 03:37:23.115256 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:23.115221    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:23.276433 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:23.276396    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:23.332320 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:23.332291    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:24.056729 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:24.056688    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:24.061987 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:24.061964    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7cb5995774-8lqs7"
Jul 10 03:37:24.276907 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:24.276873    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:24.372677 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:24.372631    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:24.472756 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:24.472723    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:24.877245 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:24.877204    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dc757fcd9-xn7db"]
Jul 10 03:37:25.502061 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:25.502027    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:25.602538 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:25.602503    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:25.988211 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:25.988176    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:37:27.969243 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:27.969208    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:28.069967 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:28.069931    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:30.645144 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:30.645108    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:31.236992 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:31.236959    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:31.338813 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:31.338778    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:34.524326 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:34.524285    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:34.591633 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:34.591600    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:36.307270 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:36.307233    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:36.410383 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:36.410349    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:36.507935 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:36.507905    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:38.858259 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:38.858207    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:38.858701 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:38.858315    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:39.099217 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:39.099184    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:39.158497 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:39.158408    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:45.311020 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.310965    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6ff58cd966-zl8zq" podUID="6abe9fca-96f7-4264-a59f-b098d2d94486" containerName="console" containerID="cri-o://c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa" gracePeriod=15
Jul 10 03:37:45.592569 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.592534    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:37:45.643425 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.643396    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6abe9fca-96f7-4264-a59f-b098d2d94486-console-serving-cert\") pod \"6abe9fca-96f7-4264-a59f-b098d2d94486\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") "
Jul 10 03:37:45.643597 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.643459    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-service-ca\") pod \"6abe9fca-96f7-4264-a59f-b098d2d94486\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") "
Jul 10 03:37:45.643597 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.643492    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6abe9fca-96f7-4264-a59f-b098d2d94486-console-oauth-config\") pod \"6abe9fca-96f7-4264-a59f-b098d2d94486\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") "
Jul 10 03:37:45.643597 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.643518    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-console-config\") pod \"6abe9fca-96f7-4264-a59f-b098d2d94486\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") "
Jul 10 03:37:45.643597 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.643534    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkj2c\" (UniqueName: \"kubernetes.io/projected/6abe9fca-96f7-4264-a59f-b098d2d94486-kube-api-access-qkj2c\") pod \"6abe9fca-96f7-4264-a59f-b098d2d94486\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") "
Jul 10 03:37:45.643597 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.643550    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-oauth-serving-cert\") pod \"6abe9fca-96f7-4264-a59f-b098d2d94486\" (UID: \"6abe9fca-96f7-4264-a59f-b098d2d94486\") "
Jul 10 03:37:45.643931 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.643905    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-service-ca" (OuterVolumeSpecName: "service-ca") pod "6abe9fca-96f7-4264-a59f-b098d2d94486" (UID: "6abe9fca-96f7-4264-a59f-b098d2d94486"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:37:45.643996 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.643973    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-console-config" (OuterVolumeSpecName: "console-config") pod "6abe9fca-96f7-4264-a59f-b098d2d94486" (UID: "6abe9fca-96f7-4264-a59f-b098d2d94486"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:37:45.644039 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.643984    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6abe9fca-96f7-4264-a59f-b098d2d94486" (UID: "6abe9fca-96f7-4264-a59f-b098d2d94486"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:37:45.645796 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.645770    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6abe9fca-96f7-4264-a59f-b098d2d94486-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6abe9fca-96f7-4264-a59f-b098d2d94486" (UID: "6abe9fca-96f7-4264-a59f-b098d2d94486"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:37:45.645896 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.645798    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6abe9fca-96f7-4264-a59f-b098d2d94486-kube-api-access-qkj2c" (OuterVolumeSpecName: "kube-api-access-qkj2c") pod "6abe9fca-96f7-4264-a59f-b098d2d94486" (UID: "6abe9fca-96f7-4264-a59f-b098d2d94486"). InnerVolumeSpecName "kube-api-access-qkj2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:37:45.645967 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.645941    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6abe9fca-96f7-4264-a59f-b098d2d94486-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6abe9fca-96f7-4264-a59f-b098d2d94486" (UID: "6abe9fca-96f7-4264-a59f-b098d2d94486"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:37:45.744431 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.744390    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6abe9fca-96f7-4264-a59f-b098d2d94486-console-serving-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:45.744431 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.744423    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:45.744431 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.744433    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6abe9fca-96f7-4264-a59f-b098d2d94486-console-oauth-config\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:45.744431 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.744443    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-console-config\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:45.744708 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.744453    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkj2c\" (UniqueName: \"kubernetes.io/projected/6abe9fca-96f7-4264-a59f-b098d2d94486-kube-api-access-qkj2c\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:45.744708 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:45.744463    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6abe9fca-96f7-4264-a59f-b098d2d94486-oauth-serving-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:46.141749 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:46.141710    2561 generic.go:356] "Generic (PLEG): container finished" podID="6abe9fca-96f7-4264-a59f-b098d2d94486" containerID="c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa" exitCode=0
Jul 10 03:37:46.141911 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:46.141761    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff58cd966-zl8zq"
Jul 10 03:37:46.141911 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:46.141792    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff58cd966-zl8zq" event={"ID":"6abe9fca-96f7-4264-a59f-b098d2d94486","Type":"ContainerDied","Data":"c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa"}
Jul 10 03:37:46.141911 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:46.141845    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff58cd966-zl8zq" event={"ID":"6abe9fca-96f7-4264-a59f-b098d2d94486","Type":"ContainerDied","Data":"582f3a32dc18b1f84b5b2b433024f90eec5b76240e89266eb667464bde32f63e"}
Jul 10 03:37:46.141911 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:46.141866    2561 scope.go:117] "RemoveContainer" containerID="c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa"
Jul 10 03:37:46.152566 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:46.152544    2561 scope.go:117] "RemoveContainer" containerID="c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa"
Jul 10 03:37:46.152960 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:37:46.152940    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa\": container with ID starting with c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa not found: ID does not exist" containerID="c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa"
Jul 10 03:37:46.153042 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:46.152972    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa"} err="failed to get container status \"c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa\": rpc error: code = NotFound desc = could not find container \"c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa\": container with ID starting with c543428f321c1577833c42966ae4dee08605bba2cf31aafa7e9ca187dbf719aa not found: ID does not exist"
Jul 10 03:37:46.164396 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:46.164370    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ff58cd966-zl8zq"]
Jul 10 03:37:46.168732 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:46.168709    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6ff58cd966-zl8zq"]
Jul 10 03:37:46.993550 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:46.993515    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6abe9fca-96f7-4264-a59f-b098d2d94486" path="/var/lib/kubelet/pods/6abe9fca-96f7-4264-a59f-b098d2d94486/volumes"
Jul 10 03:37:48.055777 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.055733    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6" podUID="32888fa2-39e4-4de1-9a19-dee518a05139" containerName="registry" containerID="cri-o://31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e" gracePeriod=30
Jul 10 03:37:48.342327 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.342292    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:37:48.366232 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.366205    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32888fa2-39e4-4de1-9a19-dee518a05139-ca-trust-extracted\") pod \"32888fa2-39e4-4de1-9a19-dee518a05139\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") "
Jul 10 03:37:48.366394 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.366258    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32888fa2-39e4-4de1-9a19-dee518a05139-image-registry-private-configuration\") pod \"32888fa2-39e4-4de1-9a19-dee518a05139\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") "
Jul 10 03:37:48.366394 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.366333    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbljc\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-kube-api-access-kbljc\") pod \"32888fa2-39e4-4de1-9a19-dee518a05139\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") "
Jul 10 03:37:48.366394 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.366361    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32888fa2-39e4-4de1-9a19-dee518a05139-trusted-ca\") pod \"32888fa2-39e4-4de1-9a19-dee518a05139\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") "
Jul 10 03:37:48.366515 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.366395    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-bound-sa-token\") pod \"32888fa2-39e4-4de1-9a19-dee518a05139\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") "
Jul 10 03:37:48.366515 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.366436    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32888fa2-39e4-4de1-9a19-dee518a05139-registry-certificates\") pod \"32888fa2-39e4-4de1-9a19-dee518a05139\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") "
Jul 10 03:37:48.366515 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.366496    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32888fa2-39e4-4de1-9a19-dee518a05139-installation-pull-secrets\") pod \"32888fa2-39e4-4de1-9a19-dee518a05139\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") "
Jul 10 03:37:48.366629 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.366518    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") pod \"32888fa2-39e4-4de1-9a19-dee518a05139\" (UID: \"32888fa2-39e4-4de1-9a19-dee518a05139\") "
Jul 10 03:37:48.367207 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.367128    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32888fa2-39e4-4de1-9a19-dee518a05139-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "32888fa2-39e4-4de1-9a19-dee518a05139" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:37:48.367613 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.367571    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32888fa2-39e4-4de1-9a19-dee518a05139-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "32888fa2-39e4-4de1-9a19-dee518a05139" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:37:48.370403 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.370376    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-kube-api-access-kbljc" (OuterVolumeSpecName: "kube-api-access-kbljc") pod "32888fa2-39e4-4de1-9a19-dee518a05139" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139"). InnerVolumeSpecName "kube-api-access-kbljc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:37:48.370493 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.370435    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "32888fa2-39e4-4de1-9a19-dee518a05139" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:37:48.370728 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.370704    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "32888fa2-39e4-4de1-9a19-dee518a05139" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:37:48.370936 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.370914    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32888fa2-39e4-4de1-9a19-dee518a05139-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "32888fa2-39e4-4de1-9a19-dee518a05139" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:37:48.371016 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.370988    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32888fa2-39e4-4de1-9a19-dee518a05139-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "32888fa2-39e4-4de1-9a19-dee518a05139" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:37:48.377057 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.377034    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32888fa2-39e4-4de1-9a19-dee518a05139-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "32888fa2-39e4-4de1-9a19-dee518a05139" (UID: "32888fa2-39e4-4de1-9a19-dee518a05139"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:37:48.468016 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.467980    2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-registry-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:48.468016 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.468010    2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32888fa2-39e4-4de1-9a19-dee518a05139-ca-trust-extracted\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:48.468016 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.468020    2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32888fa2-39e4-4de1-9a19-dee518a05139-image-registry-private-configuration\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:48.468016 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.468029    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbljc\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-kube-api-access-kbljc\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:48.468278 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.468038    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32888fa2-39e4-4de1-9a19-dee518a05139-trusted-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:48.468278 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.468047    2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32888fa2-39e4-4de1-9a19-dee518a05139-bound-sa-token\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:48.468278 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.468054    2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32888fa2-39e4-4de1-9a19-dee518a05139-registry-certificates\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:48.468278 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:48.468065    2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32888fa2-39e4-4de1-9a19-dee518a05139-installation-pull-secrets\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:49.155265 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:49.155232    2561 generic.go:356] "Generic (PLEG): container finished" podID="32888fa2-39e4-4de1-9a19-dee518a05139" containerID="31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e" exitCode=0
Jul 10 03:37:49.155674 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:49.155293    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6"
Jul 10 03:37:49.155674 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:49.155313    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6" event={"ID":"32888fa2-39e4-4de1-9a19-dee518a05139","Type":"ContainerDied","Data":"31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e"}
Jul 10 03:37:49.155674 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:49.155363    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f84d8746d-csqp6" event={"ID":"32888fa2-39e4-4de1-9a19-dee518a05139","Type":"ContainerDied","Data":"cb90d09dc96356d6b638faacfff79499e4d9fb7cd599730e86c5870b2d57e740"}
Jul 10 03:37:49.155674 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:49.155383    2561 scope.go:117] "RemoveContainer" containerID="31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e"
Jul 10 03:37:49.163549 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:49.163526    2561 scope.go:117] "RemoveContainer" containerID="31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e"
Jul 10 03:37:49.163824 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:37:49.163806    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e\": container with ID starting with 31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e not found: ID does not exist" containerID="31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e"
Jul 10 03:37:49.163880 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:49.163831    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e"} err="failed to get container status \"31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e\": rpc error: code = NotFound desc = could not find container \"31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e\": container with ID starting with 31e87cf01f2a3836908ab847a5ff1eea014382547b2243ab7d0fe2cc309e740e not found: ID does not exist"
Jul 10 03:37:49.173435 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:49.173414    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f84d8746d-csqp6"]
Jul 10 03:37:49.177603 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:49.177577    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f84d8746d-csqp6"]
Jul 10 03:37:50.080054 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.080015    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7dc757fcd9-xn7db" podUID="4e973bd5-3603-4502-86bc-097e3cc89bca" containerName="console" containerID="cri-o://87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41" gracePeriod=15
Jul 10 03:37:50.357061 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.356973    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:50.383319 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.383288    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-oauth-serving-cert\") pod \"4e973bd5-3603-4502-86bc-097e3cc89bca\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") "
Jul 10 03:37:50.383468 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.383337    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-service-ca\") pod \"4e973bd5-3603-4502-86bc-097e3cc89bca\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") "
Jul 10 03:37:50.383468 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.383371    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpvw6\" (UniqueName: \"kubernetes.io/projected/4e973bd5-3603-4502-86bc-097e3cc89bca-kube-api-access-mpvw6\") pod \"4e973bd5-3603-4502-86bc-097e3cc89bca\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") "
Jul 10 03:37:50.383468 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.383397    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e973bd5-3603-4502-86bc-097e3cc89bca-console-oauth-config\") pod \"4e973bd5-3603-4502-86bc-097e3cc89bca\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") "
Jul 10 03:37:50.383468 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.383430    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e973bd5-3603-4502-86bc-097e3cc89bca-console-serving-cert\") pod \"4e973bd5-3603-4502-86bc-097e3cc89bca\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") "
Jul 10 03:37:50.384124 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.383711    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-trusted-ca-bundle\") pod \"4e973bd5-3603-4502-86bc-097e3cc89bca\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") "
Jul 10 03:37:50.384124 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.383762    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-console-config\") pod \"4e973bd5-3603-4502-86bc-097e3cc89bca\" (UID: \"4e973bd5-3603-4502-86bc-097e3cc89bca\") "
Jul 10 03:37:50.384124 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.383860    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4e973bd5-3603-4502-86bc-097e3cc89bca" (UID: "4e973bd5-3603-4502-86bc-097e3cc89bca"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:37:50.384124 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.383982    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-service-ca" (OuterVolumeSpecName: "service-ca") pod "4e973bd5-3603-4502-86bc-097e3cc89bca" (UID: "4e973bd5-3603-4502-86bc-097e3cc89bca"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:37:50.384124 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.384076    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:50.384124 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.384097    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-oauth-serving-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:50.384124 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.384109    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4e973bd5-3603-4502-86bc-097e3cc89bca" (UID: "4e973bd5-3603-4502-86bc-097e3cc89bca"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:37:50.384612 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.384151    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-console-config" (OuterVolumeSpecName: "console-config") pod "4e973bd5-3603-4502-86bc-097e3cc89bca" (UID: "4e973bd5-3603-4502-86bc-097e3cc89bca"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:37:50.385612 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.385588    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e973bd5-3603-4502-86bc-097e3cc89bca-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4e973bd5-3603-4502-86bc-097e3cc89bca" (UID: "4e973bd5-3603-4502-86bc-097e3cc89bca"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:37:50.385724 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.385706    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e973bd5-3603-4502-86bc-097e3cc89bca-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4e973bd5-3603-4502-86bc-097e3cc89bca" (UID: "4e973bd5-3603-4502-86bc-097e3cc89bca"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:37:50.386109 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.386094    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e973bd5-3603-4502-86bc-097e3cc89bca-kube-api-access-mpvw6" (OuterVolumeSpecName: "kube-api-access-mpvw6") pod "4e973bd5-3603-4502-86bc-097e3cc89bca" (UID: "4e973bd5-3603-4502-86bc-097e3cc89bca"). InnerVolumeSpecName "kube-api-access-mpvw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:37:50.485156 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.485116    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpvw6\" (UniqueName: \"kubernetes.io/projected/4e973bd5-3603-4502-86bc-097e3cc89bca-kube-api-access-mpvw6\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:50.485156 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.485146    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e973bd5-3603-4502-86bc-097e3cc89bca-console-oauth-config\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:50.485156 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.485155    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e973bd5-3603-4502-86bc-097e3cc89bca-console-serving-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:50.485156 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.485163    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-trusted-ca-bundle\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:50.485156 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.485171    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e973bd5-3603-4502-86bc-097e3cc89bca-console-config\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:37:50.993716 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:50.993683    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32888fa2-39e4-4de1-9a19-dee518a05139" path="/var/lib/kubelet/pods/32888fa2-39e4-4de1-9a19-dee518a05139/volumes"
Jul 10 03:37:51.165123 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:51.165090    2561 generic.go:356] "Generic (PLEG): container finished" podID="4e973bd5-3603-4502-86bc-097e3cc89bca" containerID="87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41" exitCode=0
Jul 10 03:37:51.165285 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:51.165137    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc757fcd9-xn7db"
Jul 10 03:37:51.165285 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:51.165174    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc757fcd9-xn7db" event={"ID":"4e973bd5-3603-4502-86bc-097e3cc89bca","Type":"ContainerDied","Data":"87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41"}
Jul 10 03:37:51.165285 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:51.165226    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc757fcd9-xn7db" event={"ID":"4e973bd5-3603-4502-86bc-097e3cc89bca","Type":"ContainerDied","Data":"714d30b49dd1595930891c9d4d979f04dcd0d607c26a93f3b69ad09f67a77394"}
Jul 10 03:37:51.165285 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:51.165251    2561 scope.go:117] "RemoveContainer" containerID="87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41"
Jul 10 03:37:51.173076 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:51.173057    2561 scope.go:117] "RemoveContainer" containerID="87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41"
Jul 10 03:37:51.173318 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:37:51.173297    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41\": container with ID starting with 87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41 not found: ID does not exist" containerID="87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41"
Jul 10 03:37:51.173368 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:51.173328    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41"} err="failed to get container status \"87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41\": rpc error: code = NotFound desc = could not find container \"87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41\": container with ID starting with 87a161b4fc3561decd3ae8b7d4ebea3cc6183c6dd00c0c6f418fabe57154ce41 not found: ID does not exist"
Jul 10 03:37:51.181879 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:51.181855    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dc757fcd9-xn7db"]
Jul 10 03:37:51.185364 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:51.185340    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dc757fcd9-xn7db"]
Jul 10 03:37:52.993769 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:52.993740    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e973bd5-3603-4502-86bc-097e3cc89bca" path="/var/lib/kubelet/pods/4e973bd5-3603-4502-86bc-097e3cc89bca/volumes"
Jul 10 03:37:53.275942 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:53.275868    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:54.099522 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:54.099488    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:54.152546 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:54.152517    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:37:58.862425 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:58.862397    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:58.866436 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:58.866410    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7b9d8fb85f-mpsmw"
Jul 10 03:37:59.198106 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:59.198021    2561 generic.go:356] "Generic (PLEG): container finished" podID="6c312753-450d-4a92-822e-4638a7eafc6f" containerID="16f7fd2d482454ccae32f7decb8662b1b7461a11420e212ca321d4b8daa5ee6e" exitCode=0
Jul 10 03:37:59.198106 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:59.198093    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq" event={"ID":"6c312753-450d-4a92-822e-4638a7eafc6f","Type":"ContainerDied","Data":"16f7fd2d482454ccae32f7decb8662b1b7461a11420e212ca321d4b8daa5ee6e"}
Jul 10 03:37:59.198471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:59.198454    2561 scope.go:117] "RemoveContainer" containerID="16f7fd2d482454ccae32f7decb8662b1b7461a11420e212ca321d4b8daa5ee6e"
Jul 10 03:37:59.199620 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:59.199598    2561 generic.go:356] "Generic (PLEG): container finished" podID="b858fbad-d7b2-4b25-b556-c852059bdef7" containerID="8a93a685948463c31d09c69351d4889b982610c90a3c819b087261a8a3fc9240" exitCode=0
Jul 10 03:37:59.199735 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:59.199679    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-l22sz" event={"ID":"b858fbad-d7b2-4b25-b556-c852059bdef7","Type":"ContainerDied","Data":"8a93a685948463c31d09c69351d4889b982610c90a3c819b087261a8a3fc9240"}
Jul 10 03:37:59.200098 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:37:59.200084    2561 scope.go:117] "RemoveContainer" containerID="8a93a685948463c31d09c69351d4889b982610c90a3c819b087261a8a3fc9240"
Jul 10 03:38:00.205181 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:00.205146    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-l22sz" event={"ID":"b858fbad-d7b2-4b25-b556-c852059bdef7","Type":"ContainerStarted","Data":"e794fc31db5e49f2b28c15087d0b48381d438f1cafa96bdecab6f4d955120905"}
Jul 10 03:38:00.206858 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:00.206837    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-j6bzq" event={"ID":"6c312753-450d-4a92-822e-4638a7eafc6f","Type":"ContainerStarted","Data":"3b0da4e15e0d3b252c2b847bf301c363f098ccfe579a6aeb79ff4071ef443795"}
Jul 10 03:38:30.322842 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:30.322805    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:30.373847 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:30.373818    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:32.136437 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.136405    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56d469c765-zt56b"]
Jul 10 03:38:32.136813 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.136738    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6abe9fca-96f7-4264-a59f-b098d2d94486" containerName="console"
Jul 10 03:38:32.136813 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.136750    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6abe9fca-96f7-4264-a59f-b098d2d94486" containerName="console"
Jul 10 03:38:32.136813 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.136765    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e973bd5-3603-4502-86bc-097e3cc89bca" containerName="console"
Jul 10 03:38:32.136813 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.136770    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e973bd5-3603-4502-86bc-097e3cc89bca" containerName="console"
Jul 10 03:38:32.136813 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.136793    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32888fa2-39e4-4de1-9a19-dee518a05139" containerName="registry"
Jul 10 03:38:32.136813 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.136799    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="32888fa2-39e4-4de1-9a19-dee518a05139" containerName="registry"
Jul 10 03:38:32.136995 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.136841    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6abe9fca-96f7-4264-a59f-b098d2d94486" containerName="console"
Jul 10 03:38:32.136995 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.136852    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="32888fa2-39e4-4de1-9a19-dee518a05139" containerName="registry"
Jul 10 03:38:32.136995 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.136859    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e973bd5-3603-4502-86bc-097e3cc89bca" containerName="console"
Jul 10 03:38:32.141708 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.141682    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.145231 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.145206    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 10 03:38:32.145431 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.145414    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j6wc5\""
Jul 10 03:38:32.145570 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.145436    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 10 03:38:32.145570 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.145445    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 10 03:38:32.145570 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.145506    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 10 03:38:32.145755 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.145570    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 10 03:38:32.149210 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.149184    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 10 03:38:32.150174 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.150139    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56d469c765-zt56b"]
Jul 10 03:38:32.203923 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.203888    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:32.238818 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.238780    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-oauth-config\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.238818 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.238823    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-trusted-ca-bundle\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.239024 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.238847    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-config\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.239024 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.238907    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-oauth-serving-cert\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.239024 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.238938    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-service-ca\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.239024 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.238970    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-serving-cert\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.239024 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.238992    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv9kp\" (UniqueName: \"kubernetes.io/projected/0035e462-2069-481f-9900-d5f5b0fd4b9f-kube-api-access-vv9kp\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.263977 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.263951    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:32.340120 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.340091    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-trusted-ca-bundle\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.340284 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.340127    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-config\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.340284 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.340156    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-oauth-serving-cert\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.340284 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.340185    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-service-ca\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.340284 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.340225    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-serving-cert\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.340284 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.340254    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv9kp\" (UniqueName: \"kubernetes.io/projected/0035e462-2069-481f-9900-d5f5b0fd4b9f-kube-api-access-vv9kp\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.340506 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.340440    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-oauth-config\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.341017 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.340973    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-config\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.341133 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.341030    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-oauth-serving-cert\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.341133 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.341083    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-service-ca\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.341274 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.341256    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-trusted-ca-bundle\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.342918 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.342899    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-oauth-config\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.343003 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.342943    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-serving-cert\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.347480 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.347460    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv9kp\" (UniqueName: \"kubernetes.io/projected/0035e462-2069-481f-9900-d5f5b0fd4b9f-kube-api-access-vv9kp\") pod \"console-56d469c765-zt56b\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") " pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.453932 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.453846    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:32.583044 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.583017    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56d469c765-zt56b"]
Jul 10 03:38:32.585466 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:38:32.585425    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0035e462_2069_481f_9900_d5f5b0fd4b9f.slice/crio-39458bcdff6fd65f446cdbaa3ccb375d28d1c650af5d68b0bb33cfe7756f9e83 WatchSource:0}: Error finding container 39458bcdff6fd65f446cdbaa3ccb375d28d1c650af5d68b0bb33cfe7756f9e83: Status 404 returned error can't find the container with id 39458bcdff6fd65f446cdbaa3ccb375d28d1c650af5d68b0bb33cfe7756f9e83
Jul 10 03:38:32.945685 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.945623    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:38:32.948001 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:32.947971    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776389fa-994a-4f3b-bcfb-48c486232e9b-metrics-certs\") pod \"network-metrics-daemon-p7l8g\" (UID: \"776389fa-994a-4f3b-bcfb-48c486232e9b\") " pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:38:33.191512 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:33.191480    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sg5lf\""
Jul 10 03:38:33.200299 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:33.200240    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7l8g"
Jul 10 03:38:33.324550 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:33.324512    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p7l8g"]
Jul 10 03:38:33.327385 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:38:33.327358    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod776389fa_994a_4f3b_bcfb_48c486232e9b.slice/crio-bdb9e954b5679661d61e4d5d13770a2c85bd01b1810acde0194eb5c92e93f8f7 WatchSource:0}: Error finding container bdb9e954b5679661d61e4d5d13770a2c85bd01b1810acde0194eb5c92e93f8f7: Status 404 returned error can't find the container with id bdb9e954b5679661d61e4d5d13770a2c85bd01b1810acde0194eb5c92e93f8f7
Jul 10 03:38:33.331268 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:33.331246    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 03:38:33.333408 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:33.333384    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d469c765-zt56b" event={"ID":"0035e462-2069-481f-9900-d5f5b0fd4b9f","Type":"ContainerStarted","Data":"0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa"}
Jul 10 03:38:33.333502 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:33.333428    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d469c765-zt56b" event={"ID":"0035e462-2069-481f-9900-d5f5b0fd4b9f","Type":"ContainerStarted","Data":"39458bcdff6fd65f446cdbaa3ccb375d28d1c650af5d68b0bb33cfe7756f9e83"}
Jul 10 03:38:34.340149 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:34.340109    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p7l8g" event={"ID":"776389fa-994a-4f3b-bcfb-48c486232e9b","Type":"ContainerStarted","Data":"bdb9e954b5679661d61e4d5d13770a2c85bd01b1810acde0194eb5c92e93f8f7"}
Jul 10 03:38:35.345395 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:35.345356    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p7l8g" event={"ID":"776389fa-994a-4f3b-bcfb-48c486232e9b","Type":"ContainerStarted","Data":"b257f3e25ff8b210c00fab75a31670925edb7ad948d1a37a2862ee360f82a01d"}
Jul 10 03:38:35.345842 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:35.345410    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p7l8g" event={"ID":"776389fa-994a-4f3b-bcfb-48c486232e9b","Type":"ContainerStarted","Data":"ae24d166c051235e2a880e27dd557f3243708bfcd9f1d7f58ceb05f2204a7c2f"}
Jul 10 03:38:35.360924 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:35.360877    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56d469c765-zt56b" podStartSLOduration=3.360862899 podStartE2EDuration="3.360862899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:38:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:38:33.352056773 +0000 UTC m=+374.843052647" watchObservedRunningTime="2026-07-10 03:38:35.360862899 +0000 UTC m=+376.851859148"
Jul 10 03:38:35.361375 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:35.361354    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p7l8g" podStartSLOduration=375.387547537 podStartE2EDuration="6m16.361345699s" totalImagesPullingTime="973.798162ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:38:35.359790351 +0000 UTC m=+376.850786224" watchObservedRunningTime="2026-07-10 03:38:35.361345699 +0000 UTC m=+376.852341572"
Jul 10 03:38:35.454674 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:35.454616    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:35.511797 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:35.511766    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:37.344202 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:37.344168    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:37.404034 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:37.404005    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:42.341848 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:42.341813    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:42.392004 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:42.391973    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:42.454173 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:42.454144    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:42.454292 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:42.454195    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:42.459022 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:42.459001    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:43.377050 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:43.377020    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:38:43.519056 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:43.519022    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:43.568988 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:43.568923    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:43.620374 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:43.620342    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:54.109147 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:54.109110    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:54.209289 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:54.209246    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:38:54.433122 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:38:54.433030    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:14.684429 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:14.684386    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:14.780241 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:14.780206    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:15.363518 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.363489    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"]
Jul 10 03:39:15.367047 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.367026    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:15.369371 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.369349    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 10 03:39:15.369555 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.369538    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 10 03:39:15.369643 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.369611    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-czms4\""
Jul 10 03:39:15.375235 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.375210    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"]
Jul 10 03:39:15.490255 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.490223    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc\" (UID: \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:15.490420 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.490262    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc\" (UID: \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:15.490420 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.490302    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hkr9\" (UniqueName: \"kubernetes.io/projected/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-kube-api-access-4hkr9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc\" (UID: \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:15.591483 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.591446    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hkr9\" (UniqueName: \"kubernetes.io/projected/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-kube-api-access-4hkr9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc\" (UID: \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:15.591676 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.591501    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc\" (UID: \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:15.591676 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.591528    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc\" (UID: \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:15.591875 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.591861    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc\" (UID: \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:15.591915 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.591890    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc\" (UID: \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:15.599480 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.599456    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hkr9\" (UniqueName: \"kubernetes.io/projected/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-kube-api-access-4hkr9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc\" (UID: \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:15.677477 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.677378    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:15.831954 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:15.831919    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"]
Jul 10 03:39:15.835127 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:39:15.835099    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe4f6d63_cdba_4bae_9aab_9c2431ab87e1.slice/crio-b78aaefa872f122c58557b0d30d6d68b0181e361d1ea970dd4f9933c6da7fe82 WatchSource:0}: Error finding container b78aaefa872f122c58557b0d30d6d68b0181e361d1ea970dd4f9933c6da7fe82: Status 404 returned error can't find the container with id b78aaefa872f122c58557b0d30d6d68b0181e361d1ea970dd4f9933c6da7fe82
Jul 10 03:39:16.496353 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:16.496309    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc" event={"ID":"be4f6d63-cdba-4bae-9aab-9c2431ab87e1","Type":"ContainerStarted","Data":"b78aaefa872f122c58557b0d30d6d68b0181e361d1ea970dd4f9933c6da7fe82"}
Jul 10 03:39:22.524292 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:22.524262    2561 generic.go:356] "Generic (PLEG): container finished" podID="be4f6d63-cdba-4bae-9aab-9c2431ab87e1" containerID="db8ae3921e5d7943b3c2aa855f6a1858aa828d380e9e318a2228403337400349" exitCode=0
Jul 10 03:39:22.524715 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:22.524354    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc" event={"ID":"be4f6d63-cdba-4bae-9aab-9c2431ab87e1","Type":"ContainerDied","Data":"db8ae3921e5d7943b3c2aa855f6a1858aa828d380e9e318a2228403337400349"}
Jul 10 03:39:27.544473 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:27.544433    2561 generic.go:356] "Generic (PLEG): container finished" podID="be4f6d63-cdba-4bae-9aab-9c2431ab87e1" containerID="2996dc8c741652e0f8ec1e074d3188b1b3bb9ba05f364b53f8acc54f85bc75db" exitCode=0
Jul 10 03:39:27.544931 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:27.544516    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc" event={"ID":"be4f6d63-cdba-4bae-9aab-9c2431ab87e1","Type":"ContainerDied","Data":"2996dc8c741652e0f8ec1e074d3188b1b3bb9ba05f364b53f8acc54f85bc75db"}
Jul 10 03:39:30.355246 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:30.355194    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:30.420125 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:30.420084    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:33.571960 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:33.571934    2561 generic.go:356] "Generic (PLEG): container finished" podID="be4f6d63-cdba-4bae-9aab-9c2431ab87e1" containerID="abf41ff688dfc5e26308dedf7104365e3375cdfbf883ce0cf7db86be696ae70a" exitCode=0
Jul 10 03:39:33.572368 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:33.572018    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc" event={"ID":"be4f6d63-cdba-4bae-9aab-9c2431ab87e1","Type":"ContainerDied","Data":"abf41ff688dfc5e26308dedf7104365e3375cdfbf883ce0cf7db86be696ae70a"}
Jul 10 03:39:34.712357 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:34.712327    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:34.742257 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:34.742230    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-bundle\") pod \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\" (UID: \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\") "
Jul 10 03:39:34.742401 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:34.742279    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hkr9\" (UniqueName: \"kubernetes.io/projected/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-kube-api-access-4hkr9\") pod \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\" (UID: \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\") "
Jul 10 03:39:34.742401 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:34.742351    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-util\") pod \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\" (UID: \"be4f6d63-cdba-4bae-9aab-9c2431ab87e1\") "
Jul 10 03:39:34.742947 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:34.742917    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-bundle" (OuterVolumeSpecName: "bundle") pod "be4f6d63-cdba-4bae-9aab-9c2431ab87e1" (UID: "be4f6d63-cdba-4bae-9aab-9c2431ab87e1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:39:34.744508 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:34.744485    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-kube-api-access-4hkr9" (OuterVolumeSpecName: "kube-api-access-4hkr9") pod "be4f6d63-cdba-4bae-9aab-9c2431ab87e1" (UID: "be4f6d63-cdba-4bae-9aab-9c2431ab87e1"). InnerVolumeSpecName "kube-api-access-4hkr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:39:34.746492 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:34.746468    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-util" (OuterVolumeSpecName: "util") pod "be4f6d63-cdba-4bae-9aab-9c2431ab87e1" (UID: "be4f6d63-cdba-4bae-9aab-9c2431ab87e1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:39:34.843847 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:34.843809    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-util\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:39:34.844009 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:34.843861    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-bundle\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:39:34.844009 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:34.843873    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hkr9\" (UniqueName: \"kubernetes.io/projected/be4f6d63-cdba-4bae-9aab-9c2431ab87e1-kube-api-access-4hkr9\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:39:35.581897 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:35.581859    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc" event={"ID":"be4f6d63-cdba-4bae-9aab-9c2431ab87e1","Type":"ContainerDied","Data":"b78aaefa872f122c58557b0d30d6d68b0181e361d1ea970dd4f9933c6da7fe82"}
Jul 10 03:39:35.581897 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:35.581893    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qfnrc"
Jul 10 03:39:35.582102 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:35.581912    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b78aaefa872f122c58557b0d30d6d68b0181e361d1ea970dd4f9933c6da7fe82"
Jul 10 03:39:41.738422 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:41.738323    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:42.036818 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:42.036727    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:44.017799 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.017767    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq"]
Jul 10 03:39:44.018192 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.018047    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be4f6d63-cdba-4bae-9aab-9c2431ab87e1" containerName="pull"
Jul 10 03:39:44.018192 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.018057    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="be4f6d63-cdba-4bae-9aab-9c2431ab87e1" containerName="pull"
Jul 10 03:39:44.018192 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.018075    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be4f6d63-cdba-4bae-9aab-9c2431ab87e1" containerName="util"
Jul 10 03:39:44.018192 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.018080    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="be4f6d63-cdba-4bae-9aab-9c2431ab87e1" containerName="util"
Jul 10 03:39:44.018192 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.018091    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be4f6d63-cdba-4bae-9aab-9c2431ab87e1" containerName="extract"
Jul 10 03:39:44.018192 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.018097    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="be4f6d63-cdba-4bae-9aab-9c2431ab87e1" containerName="extract"
Jul 10 03:39:44.018192 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.018156    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="be4f6d63-cdba-4bae-9aab-9c2431ab87e1" containerName="extract"
Jul 10 03:39:44.021142 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.021125    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq"
Jul 10 03:39:44.023893 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.023868    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 10 03:39:44.024026 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.023900    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 10 03:39:44.024026 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.023931    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-9zshn\""
Jul 10 03:39:44.039842 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.039818    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq"]
Jul 10 03:39:44.114969 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.114919    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk7ht\" (UniqueName: \"kubernetes.io/projected/65807d82-4f48-4c1d-a991-6f19a2d41cb3-kube-api-access-pk7ht\") pod \"cert-manager-operator-controller-manager-55c444bf86-vr8mq\" (UID: \"65807d82-4f48-4c1d-a991-6f19a2d41cb3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq"
Jul 10 03:39:44.114969 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.114970    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65807d82-4f48-4c1d-a991-6f19a2d41cb3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vr8mq\" (UID: \"65807d82-4f48-4c1d-a991-6f19a2d41cb3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq"
Jul 10 03:39:44.216039 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.215990    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk7ht\" (UniqueName: \"kubernetes.io/projected/65807d82-4f48-4c1d-a991-6f19a2d41cb3-kube-api-access-pk7ht\") pod \"cert-manager-operator-controller-manager-55c444bf86-vr8mq\" (UID: \"65807d82-4f48-4c1d-a991-6f19a2d41cb3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq"
Jul 10 03:39:44.216039 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.216035    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65807d82-4f48-4c1d-a991-6f19a2d41cb3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vr8mq\" (UID: \"65807d82-4f48-4c1d-a991-6f19a2d41cb3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq"
Jul 10 03:39:44.216382 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.216364    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65807d82-4f48-4c1d-a991-6f19a2d41cb3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vr8mq\" (UID: \"65807d82-4f48-4c1d-a991-6f19a2d41cb3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq"
Jul 10 03:39:44.229379 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.229344    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk7ht\" (UniqueName: \"kubernetes.io/projected/65807d82-4f48-4c1d-a991-6f19a2d41cb3-kube-api-access-pk7ht\") pod \"cert-manager-operator-controller-manager-55c444bf86-vr8mq\" (UID: \"65807d82-4f48-4c1d-a991-6f19a2d41cb3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq"
Jul 10 03:39:44.329986 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.329954    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq"
Jul 10 03:39:44.465967 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.465928    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq"]
Jul 10 03:39:44.470175 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:39:44.470128    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65807d82_4f48_4c1d_a991_6f19a2d41cb3.slice/crio-01dabf6543c72be3419446a76baf0507bd36997e93863d894e2049cd4a47d993 WatchSource:0}: Error finding container 01dabf6543c72be3419446a76baf0507bd36997e93863d894e2049cd4a47d993: Status 404 returned error can't find the container with id 01dabf6543c72be3419446a76baf0507bd36997e93863d894e2049cd4a47d993
Jul 10 03:39:44.617245 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:44.617163    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq" event={"ID":"65807d82-4f48-4c1d-a991-6f19a2d41cb3","Type":"ContainerStarted","Data":"01dabf6543c72be3419446a76baf0507bd36997e93863d894e2049cd4a47d993"}
Jul 10 03:39:45.479548 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:45.479453    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:45.622339 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:45.622300    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:46.627942 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:46.627900    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq" event={"ID":"65807d82-4f48-4c1d-a991-6f19a2d41cb3","Type":"ContainerStarted","Data":"d4c18484c79d0f8d93a046c35f2d569a6ac8ce5b22113e9a5f6ddba8859f1520"}
Jul 10 03:39:46.651445 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:46.651391    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vr8mq" podStartSLOduration=2.108249061 podStartE2EDuration="3.651375498s" totalImagesPullingTime="1.543126437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:39:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:39:46.650276792 +0000 UTC m=+448.141272686" watchObservedRunningTime="2026-07-10 03:39:46.651375498 +0000 UTC m=+448.142371371"
Jul 10 03:39:46.918964 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:46.918888    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:47.033034 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.033002    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:47.070799 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.070763    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dlx96"]
Jul 10 03:39:47.074108 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.074089    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96"
Jul 10 03:39:47.078139 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.078107    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 10 03:39:47.079206 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.079182    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-rkbpn\""
Jul 10 03:39:47.079347 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.079190    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 10 03:39:47.133810 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.133779    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dlx96"]
Jul 10 03:39:47.142998 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.142971    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csjx7\" (UniqueName: \"kubernetes.io/projected/9de48552-b4cb-48fa-aad5-705c07c31c36-kube-api-access-csjx7\") pod \"cert-manager-webhook-6486ffd997-dlx96\" (UID: \"9de48552-b4cb-48fa-aad5-705c07c31c36\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96"
Jul 10 03:39:47.143156 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.143006    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9de48552-b4cb-48fa-aad5-705c07c31c36-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dlx96\" (UID: \"9de48552-b4cb-48fa-aad5-705c07c31c36\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96"
Jul 10 03:39:47.150629 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.150602    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:47.243785 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.243687    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csjx7\" (UniqueName: \"kubernetes.io/projected/9de48552-b4cb-48fa-aad5-705c07c31c36-kube-api-access-csjx7\") pod \"cert-manager-webhook-6486ffd997-dlx96\" (UID: \"9de48552-b4cb-48fa-aad5-705c07c31c36\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96"
Jul 10 03:39:47.243785 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.243734    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9de48552-b4cb-48fa-aad5-705c07c31c36-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dlx96\" (UID: \"9de48552-b4cb-48fa-aad5-705c07c31c36\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96"
Jul 10 03:39:47.255366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.255333    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9de48552-b4cb-48fa-aad5-705c07c31c36-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dlx96\" (UID: \"9de48552-b4cb-48fa-aad5-705c07c31c36\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96"
Jul 10 03:39:47.255504 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.255394    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csjx7\" (UniqueName: \"kubernetes.io/projected/9de48552-b4cb-48fa-aad5-705c07c31c36-kube-api-access-csjx7\") pod \"cert-manager-webhook-6486ffd997-dlx96\" (UID: \"9de48552-b4cb-48fa-aad5-705c07c31c36\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96"
Jul 10 03:39:47.394802 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.394767    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96"
Jul 10 03:39:47.560021 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.559987    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dlx96"]
Jul 10 03:39:47.562568 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:39:47.562544    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9de48552_b4cb_48fa_aad5_705c07c31c36.slice/crio-a69e9988164dcde6ce1106edc88df279e37fb9ffafa61066ee77fb9d162511ff WatchSource:0}: Error finding container a69e9988164dcde6ce1106edc88df279e37fb9ffafa61066ee77fb9d162511ff: Status 404 returned error can't find the container with id a69e9988164dcde6ce1106edc88df279e37fb9ffafa61066ee77fb9d162511ff
Jul 10 03:39:47.634477 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.634438    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96" event={"ID":"9de48552-b4cb-48fa-aad5-705c07c31c36","Type":"ContainerStarted","Data":"a69e9988164dcde6ce1106edc88df279e37fb9ffafa61066ee77fb9d162511ff"}
Jul 10 03:39:47.882263 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.882225    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"]
Jul 10 03:39:47.886895 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.886877    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:47.897332 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.897309    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 10 03:39:47.897439 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.897384    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-czms4\""
Jul 10 03:39:47.907327 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.907306    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 10 03:39:47.910995 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.910970    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"]
Jul 10 03:39:47.948513 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.948485    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd8a1868-10e4-4139-bb31-440bcd5e50d3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5\" (UID: \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:47.948670 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.948523    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t2zf\" (UniqueName: \"kubernetes.io/projected/cd8a1868-10e4-4139-bb31-440bcd5e50d3-kube-api-access-4t2zf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5\" (UID: \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:47.948670 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:47.948615    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd8a1868-10e4-4139-bb31-440bcd5e50d3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5\" (UID: \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:48.050006 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:48.049966    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd8a1868-10e4-4139-bb31-440bcd5e50d3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5\" (UID: \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:48.050147 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:48.050016    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd8a1868-10e4-4139-bb31-440bcd5e50d3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5\" (UID: \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:48.050207 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:48.050136    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t2zf\" (UniqueName: \"kubernetes.io/projected/cd8a1868-10e4-4139-bb31-440bcd5e50d3-kube-api-access-4t2zf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5\" (UID: \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:48.050361 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:48.050346    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd8a1868-10e4-4139-bb31-440bcd5e50d3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5\" (UID: \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:48.050401 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:48.050377    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd8a1868-10e4-4139-bb31-440bcd5e50d3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5\" (UID: \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:48.064542 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:48.064513    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t2zf\" (UniqueName: \"kubernetes.io/projected/cd8a1868-10e4-4139-bb31-440bcd5e50d3-kube-api-access-4t2zf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5\" (UID: \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:48.195822 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:48.195740    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:48.332452 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:48.332426    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"]
Jul 10 03:39:48.334807 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:39:48.334771    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd8a1868_10e4_4139_bb31_440bcd5e50d3.slice/crio-cf5502684edef0db81e3f576d37b2e9b2fa883041747f22e74717012b6b1eda7 WatchSource:0}: Error finding container cf5502684edef0db81e3f576d37b2e9b2fa883041747f22e74717012b6b1eda7: Status 404 returned error can't find the container with id cf5502684edef0db81e3f576d37b2e9b2fa883041747f22e74717012b6b1eda7
Jul 10 03:39:48.641304 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:48.641272    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd8a1868-10e4-4139-bb31-440bcd5e50d3" containerID="e6987c99026e07c452f18064bfb7065a1876a46cb366da67f88a17622016c215" exitCode=0
Jul 10 03:39:48.641769 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:48.641370    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5" event={"ID":"cd8a1868-10e4-4139-bb31-440bcd5e50d3","Type":"ContainerDied","Data":"e6987c99026e07c452f18064bfb7065a1876a46cb366da67f88a17622016c215"}
Jul 10 03:39:48.641769 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:48.641414    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5" event={"ID":"cd8a1868-10e4-4139-bb31-440bcd5e50d3","Type":"ContainerStarted","Data":"cf5502684edef0db81e3f576d37b2e9b2fa883041747f22e74717012b6b1eda7"}
Jul 10 03:39:49.314020 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:49.313990    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:49.475888 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:49.475850    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:50.662705 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:50.662666    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc"]
Jul 10 03:39:50.666148 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:50.666130    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc"
Jul 10 03:39:50.669360 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:50.669338    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-h7dgg\""
Jul 10 03:39:50.681872 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:50.681847    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc"]
Jul 10 03:39:50.772991 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:50.772952    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e0452ea3-116f-4314-8607-e489aac9656a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sbhnc\" (UID: \"e0452ea3-116f-4314-8607-e489aac9656a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc"
Jul 10 03:39:50.773167 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:50.773008    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcmlm\" (UniqueName: \"kubernetes.io/projected/e0452ea3-116f-4314-8607-e489aac9656a-kube-api-access-jcmlm\") pod \"cert-manager-cainjector-559cdd8c87-sbhnc\" (UID: \"e0452ea3-116f-4314-8607-e489aac9656a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc"
Jul 10 03:39:50.874131 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:50.874095    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e0452ea3-116f-4314-8607-e489aac9656a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sbhnc\" (UID: \"e0452ea3-116f-4314-8607-e489aac9656a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc"
Jul 10 03:39:50.874319 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:50.874143    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcmlm\" (UniqueName: \"kubernetes.io/projected/e0452ea3-116f-4314-8607-e489aac9656a-kube-api-access-jcmlm\") pod \"cert-manager-cainjector-559cdd8c87-sbhnc\" (UID: \"e0452ea3-116f-4314-8607-e489aac9656a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc"
Jul 10 03:39:50.884536 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:50.884473    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e0452ea3-116f-4314-8607-e489aac9656a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sbhnc\" (UID: \"e0452ea3-116f-4314-8607-e489aac9656a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc"
Jul 10 03:39:50.884769 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:50.884746    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcmlm\" (UniqueName: \"kubernetes.io/projected/e0452ea3-116f-4314-8607-e489aac9656a-kube-api-access-jcmlm\") pod \"cert-manager-cainjector-559cdd8c87-sbhnc\" (UID: \"e0452ea3-116f-4314-8607-e489aac9656a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc"
Jul 10 03:39:50.976734 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:50.976631    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc"
Jul 10 03:39:51.658620 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:51.658584    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5" event={"ID":"cd8a1868-10e4-4139-bb31-440bcd5e50d3","Type":"ContainerStarted","Data":"f8698fb8b95756e0315c0dd77bfbf48adff1f3b57ca14aef8f07d57575097c05"}
Jul 10 03:39:51.660200 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:51.660167    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96" event={"ID":"9de48552-b4cb-48fa-aad5-705c07c31c36","Type":"ContainerStarted","Data":"49f3a95b46c9717ba6f647eafc8b21047ce1f4368993c71186c28f5e0cde2c42"}
Jul 10 03:39:51.660364 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:51.660349    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96"
Jul 10 03:39:51.671394 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:51.671319    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc"]
Jul 10 03:39:51.675939 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:39:51.675899    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0452ea3_116f_4314_8607_e489aac9656a.slice/crio-fabc495ca4ed9d477fc9357dd084bedaa9908a0b4013db6e7876d305f2d8e303 WatchSource:0}: Error finding container fabc495ca4ed9d477fc9357dd084bedaa9908a0b4013db6e7876d305f2d8e303: Status 404 returned error can't find the container with id fabc495ca4ed9d477fc9357dd084bedaa9908a0b4013db6e7876d305f2d8e303
Jul 10 03:39:51.695363 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:51.695311    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96" podStartSLOduration=0.721913818 podStartE2EDuration="4.695293884s" totalImagesPullingTime="3.973380066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:39:51.694427344 +0000 UTC m=+453.185423233" watchObservedRunningTime="2026-07-10 03:39:51.695293884 +0000 UTC m=+453.186289757"
Jul 10 03:39:52.268004 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:52.267968    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:52.324073 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:52.324045    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:52.666958 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:52.666928    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd8a1868-10e4-4139-bb31-440bcd5e50d3" containerID="f8698fb8b95756e0315c0dd77bfbf48adff1f3b57ca14aef8f07d57575097c05" exitCode=0
Jul 10 03:39:52.667161 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:52.667009    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5" event={"ID":"cd8a1868-10e4-4139-bb31-440bcd5e50d3","Type":"ContainerDied","Data":"f8698fb8b95756e0315c0dd77bfbf48adff1f3b57ca14aef8f07d57575097c05"}
Jul 10 03:39:52.668971 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:52.668945    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc" event={"ID":"e0452ea3-116f-4314-8607-e489aac9656a","Type":"ContainerStarted","Data":"d397baff105360c55c4b3b9405bc9a5dfb6746f05c5e58d6d5b81a46680b1944"}
Jul 10 03:39:52.669061 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:52.668987    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc" event={"ID":"e0452ea3-116f-4314-8607-e489aac9656a","Type":"ContainerStarted","Data":"fabc495ca4ed9d477fc9357dd084bedaa9908a0b4013db6e7876d305f2d8e303"}
Jul 10 03:39:52.708147 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:52.708096    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sbhnc" podStartSLOduration=2.708080554 podStartE2EDuration="2.708080554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:39:52.707127574 +0000 UTC m=+454.198123448" watchObservedRunningTime="2026-07-10 03:39:52.708080554 +0000 UTC m=+454.199076427"
Jul 10 03:39:53.674748 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:53.674712    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd8a1868-10e4-4139-bb31-440bcd5e50d3" containerID="a2c136763bb454d46220d7dd06abb9fc68297e71286cf552ca6115729715d486" exitCode=0
Jul 10 03:39:53.674924 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:53.674803    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5" event={"ID":"cd8a1868-10e4-4139-bb31-440bcd5e50d3","Type":"ContainerDied","Data":"a2c136763bb454d46220d7dd06abb9fc68297e71286cf552ca6115729715d486"}
Jul 10 03:39:54.812506 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:54.812480    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:54.909096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:54.909056    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd8a1868-10e4-4139-bb31-440bcd5e50d3-bundle\") pod \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\" (UID: \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\") "
Jul 10 03:39:54.909278 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:54.909166    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd8a1868-10e4-4139-bb31-440bcd5e50d3-util\") pod \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\" (UID: \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\") "
Jul 10 03:39:54.909278 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:54.909220    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t2zf\" (UniqueName: \"kubernetes.io/projected/cd8a1868-10e4-4139-bb31-440bcd5e50d3-kube-api-access-4t2zf\") pod \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\" (UID: \"cd8a1868-10e4-4139-bb31-440bcd5e50d3\") "
Jul 10 03:39:54.909513 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:54.909485    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd8a1868-10e4-4139-bb31-440bcd5e50d3-bundle" (OuterVolumeSpecName: "bundle") pod "cd8a1868-10e4-4139-bb31-440bcd5e50d3" (UID: "cd8a1868-10e4-4139-bb31-440bcd5e50d3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:39:54.911340 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:54.911320    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd8a1868-10e4-4139-bb31-440bcd5e50d3-kube-api-access-4t2zf" (OuterVolumeSpecName: "kube-api-access-4t2zf") pod "cd8a1868-10e4-4139-bb31-440bcd5e50d3" (UID: "cd8a1868-10e4-4139-bb31-440bcd5e50d3"). InnerVolumeSpecName "kube-api-access-4t2zf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:39:54.913902 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:54.913877    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd8a1868-10e4-4139-bb31-440bcd5e50d3-util" (OuterVolumeSpecName: "util") pod "cd8a1868-10e4-4139-bb31-440bcd5e50d3" (UID: "cd8a1868-10e4-4139-bb31-440bcd5e50d3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:39:55.010062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:55.010032    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd8a1868-10e4-4139-bb31-440bcd5e50d3-util\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:39:55.010062 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:55.010059    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t2zf\" (UniqueName: \"kubernetes.io/projected/cd8a1868-10e4-4139-bb31-440bcd5e50d3-kube-api-access-4t2zf\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:39:55.010218 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:55.010070    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd8a1868-10e4-4139-bb31-440bcd5e50d3-bundle\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:39:55.690019 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:55.689984    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5" event={"ID":"cd8a1868-10e4-4139-bb31-440bcd5e50d3","Type":"ContainerDied","Data":"cf5502684edef0db81e3f576d37b2e9b2fa883041747f22e74717012b6b1eda7"}
Jul 10 03:39:55.690191 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:55.690036    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf5502684edef0db81e3f576d37b2e9b2fa883041747f22e74717012b6b1eda7"
Jul 10 03:39:55.690191 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:55.689998    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgqvh5"
Jul 10 03:39:56.865518 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:56.865490    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:57.672060 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:57.672028    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dlx96"
Jul 10 03:39:58.620932 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.620890    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xp4x7"]
Jul 10 03:39:58.621397 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.621326    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd8a1868-10e4-4139-bb31-440bcd5e50d3" containerName="pull"
Jul 10 03:39:58.621397 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.621344    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd8a1868-10e4-4139-bb31-440bcd5e50d3" containerName="pull"
Jul 10 03:39:58.621397 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.621368    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd8a1868-10e4-4139-bb31-440bcd5e50d3" containerName="extract"
Jul 10 03:39:58.621397 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.621376    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd8a1868-10e4-4139-bb31-440bcd5e50d3" containerName="extract"
Jul 10 03:39:58.621397 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.621389    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd8a1868-10e4-4139-bb31-440bcd5e50d3" containerName="util"
Jul 10 03:39:58.621397 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.621397    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd8a1868-10e4-4139-bb31-440bcd5e50d3" containerName="util"
Jul 10 03:39:58.621744 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.621483    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd8a1868-10e4-4139-bb31-440bcd5e50d3" containerName="extract"
Jul 10 03:39:58.625116 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.625095    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xp4x7"
Jul 10 03:39:58.627203 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.627183    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-h72ns\""
Jul 10 03:39:58.636208 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.636183    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xp4x7"]
Jul 10 03:39:58.741357 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.741325    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64k74\" (UniqueName: \"kubernetes.io/projected/cb817e31-4e2e-4459-acc4-436a90f68c08-kube-api-access-64k74\") pod \"cert-manager-55b4d954d5-xp4x7\" (UID: \"cb817e31-4e2e-4459-acc4-436a90f68c08\") " pod="cert-manager/cert-manager-55b4d954d5-xp4x7"
Jul 10 03:39:58.741357 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.741364    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb817e31-4e2e-4459-acc4-436a90f68c08-bound-sa-token\") pod \"cert-manager-55b4d954d5-xp4x7\" (UID: \"cb817e31-4e2e-4459-acc4-436a90f68c08\") " pod="cert-manager/cert-manager-55b4d954d5-xp4x7"
Jul 10 03:39:58.842822 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.842772    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64k74\" (UniqueName: \"kubernetes.io/projected/cb817e31-4e2e-4459-acc4-436a90f68c08-kube-api-access-64k74\") pod \"cert-manager-55b4d954d5-xp4x7\" (UID: \"cb817e31-4e2e-4459-acc4-436a90f68c08\") " pod="cert-manager/cert-manager-55b4d954d5-xp4x7"
Jul 10 03:39:58.842822 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.842825    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb817e31-4e2e-4459-acc4-436a90f68c08-bound-sa-token\") pod \"cert-manager-55b4d954d5-xp4x7\" (UID: \"cb817e31-4e2e-4459-acc4-436a90f68c08\") " pod="cert-manager/cert-manager-55b4d954d5-xp4x7"
Jul 10 03:39:58.852707 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.852677    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64k74\" (UniqueName: \"kubernetes.io/projected/cb817e31-4e2e-4459-acc4-436a90f68c08-kube-api-access-64k74\") pod \"cert-manager-55b4d954d5-xp4x7\" (UID: \"cb817e31-4e2e-4459-acc4-436a90f68c08\") " pod="cert-manager/cert-manager-55b4d954d5-xp4x7"
Jul 10 03:39:58.852885 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.852863    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb817e31-4e2e-4459-acc4-436a90f68c08-bound-sa-token\") pod \"cert-manager-55b4d954d5-xp4x7\" (UID: \"cb817e31-4e2e-4459-acc4-436a90f68c08\") " pod="cert-manager/cert-manager-55b4d954d5-xp4x7"
Jul 10 03:39:58.933581 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:58.933498    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xp4x7"
Jul 10 03:39:59.063138 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:59.063104    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xp4x7"]
Jul 10 03:39:59.066118 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:39:59.066091    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb817e31_4e2e_4459_acc4_436a90f68c08.slice/crio-efac8fc49157306ec73aff832fd61978a0c4c0c5a838de4c4754cd3d3ed45d3a WatchSource:0}: Error finding container efac8fc49157306ec73aff832fd61978a0c4c0c5a838de4c4754cd3d3ed45d3a: Status 404 returned error can't find the container with id efac8fc49157306ec73aff832fd61978a0c4c0c5a838de4c4754cd3d3ed45d3a
Jul 10 03:39:59.536384 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:59.536345    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:59.636229 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:59.636193    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:39:59.710732 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:59.710690    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xp4x7" event={"ID":"cb817e31-4e2e-4459-acc4-436a90f68c08","Type":"ContainerStarted","Data":"986728c28e3b5f28e707e838901826fbd2e28071ce473d6b059e3b175feed1be"}
Jul 10 03:39:59.710880 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:59.710742    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xp4x7" event={"ID":"cb817e31-4e2e-4459-acc4-436a90f68c08","Type":"ContainerStarted","Data":"efac8fc49157306ec73aff832fd61978a0c4c0c5a838de4c4754cd3d3ed45d3a"}
Jul 10 03:39:59.725487 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:39:59.725293    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-xp4x7" podStartSLOduration=1.725272712 podStartE2EDuration="1.725272712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:39:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:39:59.724507145 +0000 UTC m=+461.215503035" watchObservedRunningTime="2026-07-10 03:39:59.725272712 +0000 UTC m=+461.216268585"
Jul 10 03:40:06.396534 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:06.396496    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:06.492739 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:06.492683    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:08.171810 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.171774    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"]
Jul 10 03:40:08.175560 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.175533    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:08.177926 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.177901    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 10 03:40:08.178712 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.178697    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-czms4\""
Jul 10 03:40:08.178766 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.178745    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 10 03:40:08.182201 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.182166    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"]
Jul 10 03:40:08.324300 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.324263    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/72973950-1cbc-43a2-b53d-6ad5f2d6b667-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt\" (UID: \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:08.324482 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.324326    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/72973950-1cbc-43a2-b53d-6ad5f2d6b667-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt\" (UID: \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:08.324482 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.324363    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbvbl\" (UniqueName: \"kubernetes.io/projected/72973950-1cbc-43a2-b53d-6ad5f2d6b667-kube-api-access-sbvbl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt\" (UID: \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:08.425600 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.425505    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbvbl\" (UniqueName: \"kubernetes.io/projected/72973950-1cbc-43a2-b53d-6ad5f2d6b667-kube-api-access-sbvbl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt\" (UID: \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:08.425600 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.425566    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/72973950-1cbc-43a2-b53d-6ad5f2d6b667-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt\" (UID: \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:08.425862 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.425611    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/72973950-1cbc-43a2-b53d-6ad5f2d6b667-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt\" (UID: \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:08.426035 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.426011    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/72973950-1cbc-43a2-b53d-6ad5f2d6b667-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt\" (UID: \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:08.426035 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.426024    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/72973950-1cbc-43a2-b53d-6ad5f2d6b667-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt\" (UID: \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:08.434528 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.434505    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbvbl\" (UniqueName: \"kubernetes.io/projected/72973950-1cbc-43a2-b53d-6ad5f2d6b667-kube-api-access-sbvbl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt\" (UID: \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:08.486491 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.486449    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:08.611114 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.611063    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:08.623228 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.623206    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"]
Jul 10 03:40:08.625718 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:40:08.625690    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72973950_1cbc_43a2_b53d_6ad5f2d6b667.slice/crio-29ed152e720ddcbf4f1991d7432fdf93ddd04afd435aa0d8b41590715c748b30 WatchSource:0}: Error finding container 29ed152e720ddcbf4f1991d7432fdf93ddd04afd435aa0d8b41590715c748b30: Status 404 returned error can't find the container with id 29ed152e720ddcbf4f1991d7432fdf93ddd04afd435aa0d8b41590715c748b30
Jul 10 03:40:08.752399 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.752367    2561 generic.go:356] "Generic (PLEG): container finished" podID="72973950-1cbc-43a2-b53d-6ad5f2d6b667" containerID="f89fba2bc7d0b526a3fd3ee5b5cabde34101ddb82b7152dd20c0b7c77ed7a652" exitCode=0
Jul 10 03:40:08.752577 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.752455    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt" event={"ID":"72973950-1cbc-43a2-b53d-6ad5f2d6b667","Type":"ContainerDied","Data":"f89fba2bc7d0b526a3fd3ee5b5cabde34101ddb82b7152dd20c0b7c77ed7a652"}
Jul 10 03:40:08.752577 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.752505    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt" event={"ID":"72973950-1cbc-43a2-b53d-6ad5f2d6b667","Type":"ContainerStarted","Data":"29ed152e720ddcbf4f1991d7432fdf93ddd04afd435aa0d8b41590715c748b30"}
Jul 10 03:40:08.899225 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.899187    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:08.963148 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:08.963051    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:09.759267 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:09.759180    2561 generic.go:356] "Generic (PLEG): container finished" podID="72973950-1cbc-43a2-b53d-6ad5f2d6b667" containerID="617c8dbdf46f0cad428bcbd99dd9d1de496cd2b627b666c4d66a24dfd189e5fd" exitCode=0
Jul 10 03:40:09.759627 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:09.759269    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt" event={"ID":"72973950-1cbc-43a2-b53d-6ad5f2d6b667","Type":"ContainerDied","Data":"617c8dbdf46f0cad428bcbd99dd9d1de496cd2b627b666c4d66a24dfd189e5fd"}
Jul 10 03:40:10.765556 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:10.765522    2561 generic.go:356] "Generic (PLEG): container finished" podID="72973950-1cbc-43a2-b53d-6ad5f2d6b667" containerID="1dd61888884b69eb240dcc59496c6ba39ccbeedb475f997b38b82ddf9b9857ea" exitCode=0
Jul 10 03:40:10.765940 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:10.765564    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt" event={"ID":"72973950-1cbc-43a2-b53d-6ad5f2d6b667","Type":"ContainerDied","Data":"1dd61888884b69eb240dcc59496c6ba39ccbeedb475f997b38b82ddf9b9857ea"}
Jul 10 03:40:11.900089 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:11.900059    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:12.061414 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:12.061320    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/72973950-1cbc-43a2-b53d-6ad5f2d6b667-util\") pod \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\" (UID: \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\") "
Jul 10 03:40:12.061414 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:12.061409    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbvbl\" (UniqueName: \"kubernetes.io/projected/72973950-1cbc-43a2-b53d-6ad5f2d6b667-kube-api-access-sbvbl\") pod \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\" (UID: \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\") "
Jul 10 03:40:12.061676 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:12.061437    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/72973950-1cbc-43a2-b53d-6ad5f2d6b667-bundle\") pod \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\" (UID: \"72973950-1cbc-43a2-b53d-6ad5f2d6b667\") "
Jul 10 03:40:12.062212 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:12.062166    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72973950-1cbc-43a2-b53d-6ad5f2d6b667-bundle" (OuterVolumeSpecName: "bundle") pod "72973950-1cbc-43a2-b53d-6ad5f2d6b667" (UID: "72973950-1cbc-43a2-b53d-6ad5f2d6b667"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:40:12.063440 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:12.063412    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72973950-1cbc-43a2-b53d-6ad5f2d6b667-kube-api-access-sbvbl" (OuterVolumeSpecName: "kube-api-access-sbvbl") pod "72973950-1cbc-43a2-b53d-6ad5f2d6b667" (UID: "72973950-1cbc-43a2-b53d-6ad5f2d6b667"). InnerVolumeSpecName "kube-api-access-sbvbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:40:12.067007 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:12.066983    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72973950-1cbc-43a2-b53d-6ad5f2d6b667-util" (OuterVolumeSpecName: "util") pod "72973950-1cbc-43a2-b53d-6ad5f2d6b667" (UID: "72973950-1cbc-43a2-b53d-6ad5f2d6b667"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:40:12.162533 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:12.162496    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbvbl\" (UniqueName: \"kubernetes.io/projected/72973950-1cbc-43a2-b53d-6ad5f2d6b667-kube-api-access-sbvbl\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:40:12.162533 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:12.162527    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/72973950-1cbc-43a2-b53d-6ad5f2d6b667-bundle\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:40:12.162533 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:12.162537    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/72973950-1cbc-43a2-b53d-6ad5f2d6b667-util\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:40:12.776499 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:12.776471    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt"
Jul 10 03:40:12.776693 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:12.776465    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af272jt" event={"ID":"72973950-1cbc-43a2-b53d-6ad5f2d6b667","Type":"ContainerDied","Data":"29ed152e720ddcbf4f1991d7432fdf93ddd04afd435aa0d8b41590715c748b30"}
Jul 10 03:40:12.776693 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:12.776606    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29ed152e720ddcbf4f1991d7432fdf93ddd04afd435aa0d8b41590715c748b30"
Jul 10 03:40:20.128643 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:20.128601    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:20.227207 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:20.227171    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:20.719153 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:20.719118    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:20.976070 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:20.975976    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:21.076511 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.076474    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:21.113264 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.113230    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"]
Jul 10 03:40:21.113559 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.113547    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72973950-1cbc-43a2-b53d-6ad5f2d6b667" containerName="util"
Jul 10 03:40:21.113610 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.113560    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="72973950-1cbc-43a2-b53d-6ad5f2d6b667" containerName="util"
Jul 10 03:40:21.113610 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.113571    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72973950-1cbc-43a2-b53d-6ad5f2d6b667" containerName="pull"
Jul 10 03:40:21.113610 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.113579    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="72973950-1cbc-43a2-b53d-6ad5f2d6b667" containerName="pull"
Jul 10 03:40:21.113610 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.113592    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72973950-1cbc-43a2-b53d-6ad5f2d6b667" containerName="extract"
Jul 10 03:40:21.113610 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.113598    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="72973950-1cbc-43a2-b53d-6ad5f2d6b667" containerName="extract"
Jul 10 03:40:21.113779 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.113686    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="72973950-1cbc-43a2-b53d-6ad5f2d6b667" containerName="extract"
Jul 10 03:40:21.148226 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.148191    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"]
Jul 10 03:40:21.148688 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.148318    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.150466 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.150440    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 03:40:21.150605 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.150537    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 03:40:21.150695 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.150644    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mzf2m\""
Jul 10 03:40:21.150753 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.150645    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 03:40:21.150899 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.150885    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 03:40:21.225123 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.225084    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:21.238272 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.238207    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/822c49c8-42d2-42f5-9684-b21afcdc591f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.238272 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.238249    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/822c49c8-42d2-42f5-9684-b21afcdc591f-tmp\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.238414 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.238306    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2dqc\" (UniqueName: \"kubernetes.io/projected/822c49c8-42d2-42f5-9684-b21afcdc591f-kube-api-access-h2dqc\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.238414 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.238353    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/822c49c8-42d2-42f5-9684-b21afcdc591f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.238414 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.238390    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/822c49c8-42d2-42f5-9684-b21afcdc591f-manifests\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.325845 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.325808    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:21.339583 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.339547    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2dqc\" (UniqueName: \"kubernetes.io/projected/822c49c8-42d2-42f5-9684-b21afcdc591f-kube-api-access-h2dqc\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.339720 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.339593    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/822c49c8-42d2-42f5-9684-b21afcdc591f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.339720 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.339626    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/822c49c8-42d2-42f5-9684-b21afcdc591f-manifests\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.339720 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.339672    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/822c49c8-42d2-42f5-9684-b21afcdc591f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.339828 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.339729    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/822c49c8-42d2-42f5-9684-b21afcdc591f-tmp\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.340165 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.340143    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/822c49c8-42d2-42f5-9684-b21afcdc591f-manifests\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.341916 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.341892    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/822c49c8-42d2-42f5-9684-b21afcdc591f-tmp\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.342162 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.342146    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/822c49c8-42d2-42f5-9684-b21afcdc591f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.342219 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.342172    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/822c49c8-42d2-42f5-9684-b21afcdc591f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.347687 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.347636    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2dqc\" (UniqueName: \"kubernetes.io/projected/822c49c8-42d2-42f5-9684-b21afcdc591f-kube-api-access-h2dqc\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-grr6m\" (UID: \"822c49c8-42d2-42f5-9684-b21afcdc591f\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.427281 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.427245    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:21.458114 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.458076    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:21.539073 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.538950    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:21.628394 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.628368    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"]
Jul 10 03:40:21.631235 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:40:21.631206    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod822c49c8_42d2_42f5_9684_b21afcdc591f.slice/crio-60cb5ce3a4f81b911c66f920f9b98dfbb116354845fa42a8ce16cc30fd609a2b WatchSource:0}: Error finding container 60cb5ce3a4f81b911c66f920f9b98dfbb116354845fa42a8ce16cc30fd609a2b: Status 404 returned error can't find the container with id 60cb5ce3a4f81b911c66f920f9b98dfbb116354845fa42a8ce16cc30fd609a2b
Jul 10 03:40:21.637741 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.637715    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:21.733957 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.733924    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:21.824973 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.824937    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m" event={"ID":"822c49c8-42d2-42f5-9684-b21afcdc591f","Type":"ContainerStarted","Data":"60cb5ce3a4f81b911c66f920f9b98dfbb116354845fa42a8ce16cc30fd609a2b"}
Jul 10 03:40:21.829210 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.829180    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:21.929488 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:21.929457    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:24.842500 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:24.842460    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m" event={"ID":"822c49c8-42d2-42f5-9684-b21afcdc591f","Type":"ContainerStarted","Data":"093f3921fc3c0a8fdaa5e6454878e0350c4d396e9437d2999da4b9bef0d9adf1"}
Jul 10 03:40:25.848327 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:25.848296    2561 generic.go:356] "Generic (PLEG): container finished" podID="822c49c8-42d2-42f5-9684-b21afcdc591f" containerID="093f3921fc3c0a8fdaa5e6454878e0350c4d396e9437d2999da4b9bef0d9adf1" exitCode=0
Jul 10 03:40:25.848804 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:25.848384    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m" event={"ID":"822c49c8-42d2-42f5-9684-b21afcdc591f","Type":"ContainerDied","Data":"093f3921fc3c0a8fdaa5e6454878e0350c4d396e9437d2999da4b9bef0d9adf1"}
Jul 10 03:40:26.855202 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:26.855165    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m" event={"ID":"822c49c8-42d2-42f5-9684-b21afcdc591f","Type":"ContainerStarted","Data":"6d1822a8eda9dfcb7962c754801f8a09d03d21e388b86a7c871812beb06fad75"}
Jul 10 03:40:26.855577 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:26.855284    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:26.875936 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:26.875884    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m" podStartSLOduration=1.556989076 podStartE2EDuration="5.875869148s" totalImagesPullingTime="3.22051199s" totalInitContainerRuntime="1.098368082s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:40:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:40:26.873270708 +0000 UTC m=+488.364266601" watchObservedRunningTime="2026-07-10 03:40:26.875869148 +0000 UTC m=+488.366865069"
Jul 10 03:40:27.573215 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:27.573174    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:35.091348 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:35.091310    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:35.192382 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:35.192347    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:37.861732 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:37.861695    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-grr6m"
Jul 10 03:40:39.411339 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:39.411297    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:39.516185 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:39.516148    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:40.060256 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:40.060218    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:40.116157 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:40.116124    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:40.231064 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:40.231034    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:40.640902 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:40.640860    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:40.747809 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:40.747778    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:41.125497 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.125465    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:41.396103 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.396020    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:41.545005 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.544971    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"]
Jul 10 03:40:41.553137 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.553112    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"
Jul 10 03:40:41.555315 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.555291    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 10 03:40:41.556291 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.556270    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 03:40:41.556563 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.556326    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 10 03:40:41.556749 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.556728    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 03:40:41.556917 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.556349    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2746x\""
Jul 10 03:40:41.559186 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.559162    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"]
Jul 10 03:40:41.618103 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.618059    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfca196d-b171-4d6c-8989-91ff58a6ffd5-tls-certs\") pod \"kube-auth-proxy-d9645bfdd-gmc4w\" (UID: \"dfca196d-b171-4d6c-8989-91ff58a6ffd5\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"
Jul 10 03:40:41.618103 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.618105    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfca196d-b171-4d6c-8989-91ff58a6ffd5-tmp\") pod \"kube-auth-proxy-d9645bfdd-gmc4w\" (UID: \"dfca196d-b171-4d6c-8989-91ff58a6ffd5\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"
Jul 10 03:40:41.618329 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.618134    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnqfq\" (UniqueName: \"kubernetes.io/projected/dfca196d-b171-4d6c-8989-91ff58a6ffd5-kube-api-access-pnqfq\") pod \"kube-auth-proxy-d9645bfdd-gmc4w\" (UID: \"dfca196d-b171-4d6c-8989-91ff58a6ffd5\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"
Jul 10 03:40:41.719476 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.719382    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfca196d-b171-4d6c-8989-91ff58a6ffd5-tls-certs\") pod \"kube-auth-proxy-d9645bfdd-gmc4w\" (UID: \"dfca196d-b171-4d6c-8989-91ff58a6ffd5\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"
Jul 10 03:40:41.719476 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.719431    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfca196d-b171-4d6c-8989-91ff58a6ffd5-tmp\") pod \"kube-auth-proxy-d9645bfdd-gmc4w\" (UID: \"dfca196d-b171-4d6c-8989-91ff58a6ffd5\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"
Jul 10 03:40:41.720012 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:40:41.719532    2561 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 10 03:40:41.720012 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.719562    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnqfq\" (UniqueName: \"kubernetes.io/projected/dfca196d-b171-4d6c-8989-91ff58a6ffd5-kube-api-access-pnqfq\") pod \"kube-auth-proxy-d9645bfdd-gmc4w\" (UID: \"dfca196d-b171-4d6c-8989-91ff58a6ffd5\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"
Jul 10 03:40:41.720012 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:40:41.719617    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfca196d-b171-4d6c-8989-91ff58a6ffd5-tls-certs podName:dfca196d-b171-4d6c-8989-91ff58a6ffd5 nodeName:}" failed. No retries permitted until 2026-07-10 03:40:42.219600902 +0000 UTC m=+503.710596752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dfca196d-b171-4d6c-8989-91ff58a6ffd5-tls-certs") pod "kube-auth-proxy-d9645bfdd-gmc4w" (UID: "dfca196d-b171-4d6c-8989-91ff58a6ffd5") : secret "kube-auth-proxy-tls" not found
Jul 10 03:40:41.721636 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.721616    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfca196d-b171-4d6c-8989-91ff58a6ffd5-tmp\") pod \"kube-auth-proxy-d9645bfdd-gmc4w\" (UID: \"dfca196d-b171-4d6c-8989-91ff58a6ffd5\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"
Jul 10 03:40:41.730616 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.730593    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnqfq\" (UniqueName: \"kubernetes.io/projected/dfca196d-b171-4d6c-8989-91ff58a6ffd5-kube-api-access-pnqfq\") pod \"kube-auth-proxy-d9645bfdd-gmc4w\" (UID: \"dfca196d-b171-4d6c-8989-91ff58a6ffd5\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"
Jul 10 03:40:41.932801 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:41.932768    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:42.223980 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:42.223947    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfca196d-b171-4d6c-8989-91ff58a6ffd5-tls-certs\") pod \"kube-auth-proxy-d9645bfdd-gmc4w\" (UID: \"dfca196d-b171-4d6c-8989-91ff58a6ffd5\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"
Jul 10 03:40:42.226387 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:42.226355    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfca196d-b171-4d6c-8989-91ff58a6ffd5-tls-certs\") pod \"kube-auth-proxy-d9645bfdd-gmc4w\" (UID: \"dfca196d-b171-4d6c-8989-91ff58a6ffd5\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"
Jul 10 03:40:42.465087 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:42.465052    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"
Jul 10 03:40:42.473811 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:42.473785    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:42.541998 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:42.541952    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:42.600055 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:42.600028    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w"]
Jul 10 03:40:42.602188 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:40:42.602154    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfca196d_b171_4d6c_8989_91ff58a6ffd5.slice/crio-9f66d10a71aeae9d5a9e942376db2128e60f2c743480473027494dc9ef78c94c WatchSource:0}: Error finding container 9f66d10a71aeae9d5a9e942376db2128e60f2c743480473027494dc9ef78c94c: Status 404 returned error can't find the container with id 9f66d10a71aeae9d5a9e942376db2128e60f2c743480473027494dc9ef78c94c
Jul 10 03:40:42.930360 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:42.930318    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w" event={"ID":"dfca196d-b171-4d6c-8989-91ff58a6ffd5","Type":"ContainerStarted","Data":"9f66d10a71aeae9d5a9e942376db2128e60f2c743480473027494dc9ef78c94c"}
Jul 10 03:40:43.333854 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:43.333821    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:45.946921 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:45.946884    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w" event={"ID":"dfca196d-b171-4d6c-8989-91ff58a6ffd5","Type":"ContainerStarted","Data":"450fba3308d9f5f54639db6813e7f79512cb8832d74c8457536483269a70ba00"}
Jul 10 03:40:45.962576 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:45.962524    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-d9645bfdd-gmc4w" podStartSLOduration=2.289515571 podStartE2EDuration="4.962508474s" totalImagesPullingTime="2.672992903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:40:45.961304811 +0000 UTC m=+507.452300684" watchObservedRunningTime="2026-07-10 03:40:45.962508474 +0000 UTC m=+507.453504346"
Jul 10 03:40:47.702241 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.702206    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"]
Jul 10 03:40:47.705711 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.705692    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:47.710089 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.710069    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-j9928\""
Jul 10 03:40:47.710382 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.710357    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 10 03:40:47.710382 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.710379    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 10 03:40:47.710576 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.710407    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 10 03:40:47.710576 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.710370    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 10 03:40:47.710693 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.710635    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 10 03:40:47.731693 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.731639    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"]
Jul 10 03:40:47.776028 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.775991    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4-cert\") pod \"lws-controller-manager-6549b595fc-klrpm\" (UID: \"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4\") " pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:47.776028 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.776027    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xwmx\" (UniqueName: \"kubernetes.io/projected/9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4-kube-api-access-7xwmx\") pod \"lws-controller-manager-6549b595fc-klrpm\" (UID: \"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4\") " pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:47.776246 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.776057    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4-metrics-cert\") pod \"lws-controller-manager-6549b595fc-klrpm\" (UID: \"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4\") " pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:47.776246 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.776075    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4-manager-config\") pod \"lws-controller-manager-6549b595fc-klrpm\" (UID: \"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4\") " pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:47.877390 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.877350    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4-metrics-cert\") pod \"lws-controller-manager-6549b595fc-klrpm\" (UID: \"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4\") " pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:47.877390 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.877394    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4-manager-config\") pod \"lws-controller-manager-6549b595fc-klrpm\" (UID: \"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4\") " pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:47.877641 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.877488    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4-cert\") pod \"lws-controller-manager-6549b595fc-klrpm\" (UID: \"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4\") " pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:47.877641 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.877510    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xwmx\" (UniqueName: \"kubernetes.io/projected/9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4-kube-api-access-7xwmx\") pod \"lws-controller-manager-6549b595fc-klrpm\" (UID: \"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4\") " pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:47.878185 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.878158    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4-manager-config\") pod \"lws-controller-manager-6549b595fc-klrpm\" (UID: \"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4\") " pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:47.879884 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.879861    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4-metrics-cert\") pod \"lws-controller-manager-6549b595fc-klrpm\" (UID: \"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4\") " pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:47.880044 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.880020    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4-cert\") pod \"lws-controller-manager-6549b595fc-klrpm\" (UID: \"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4\") " pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:47.889417 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:47.889357    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xwmx\" (UniqueName: \"kubernetes.io/projected/9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4-kube-api-access-7xwmx\") pod \"lws-controller-manager-6549b595fc-klrpm\" (UID: \"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4\") " pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:48.016150 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:48.016065    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:48.157831 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:48.157803    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"]
Jul 10 03:40:48.160529 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:40:48.160504    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df87cbb_2dab_483b_9ac5_d7ec4cb53fc4.slice/crio-d515c7f632a507aed1684a0beba85f432d0c4d78611efc0999c57af7c13e399f WatchSource:0}: Error finding container d515c7f632a507aed1684a0beba85f432d0c4d78611efc0999c57af7c13e399f: Status 404 returned error can't find the container with id d515c7f632a507aed1684a0beba85f432d0c4d78611efc0999c57af7c13e399f
Jul 10 03:40:48.646081 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:48.646043    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:48.778413 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:48.778375    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:48.962384 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:48.962302    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm" event={"ID":"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4","Type":"ContainerStarted","Data":"d515c7f632a507aed1684a0beba85f432d0c4d78611efc0999c57af7c13e399f"}
Jul 10 03:40:49.556191 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:49.556154    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:49.732996 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:49.732960    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:49.881165 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:49.881132    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:50.974551 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:50.974511    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm" event={"ID":"9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4","Type":"ContainerStarted","Data":"60ce6876d300d6eb179356b90168101c387398d17d280450644d24929bb676c6"}
Jul 10 03:40:50.975054 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:50.974619    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:40:51.001320 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:51.001264    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm" podStartSLOduration=1.851869253 podStartE2EDuration="4.00125052s" totalImagesPullingTime="2.149381267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:40:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:40:50.999070826 +0000 UTC m=+512.490066698" watchObservedRunningTime="2026-07-10 03:40:51.00125052 +0000 UTC m=+512.492246463"
Jul 10 03:40:51.224678 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:51.224553    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:51.327207 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:51.327161    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:51.432419 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:51.432374    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:51.549810 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:51.549725    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:51.675123 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:51.675078    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:51.773743 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:51.773685    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:40:52.451962 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:40:52.451918    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:41:01.983799 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:01.983764    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6549b595fc-klrpm"
Jul 10 03:41:14.052014 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:14.051920    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:41:16.005083 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:16.005032    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:41:16.106399 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:16.106369    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:41:26.237047 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:26.237008    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:41:26.286504 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:26.286469    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:41:40.934053 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:40.934006    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:41:40.997294 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:40.997249    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:41:41.119048 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:41.119011    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:41:49.780913 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:49.780877    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:41:56.448126 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.448094    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"]
Jul 10 03:41:56.451657 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.451627    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:41:56.454078 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.454057    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 03:41:56.454161 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.454064    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 03:41:56.454161 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.454146    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8p5ls\""
Jul 10 03:41:56.459171 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.459149    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"]
Jul 10 03:41:56.566333 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.566295    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb5e9022-8d73-46cc-9a8a-1c026c307c69-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r\" (UID: \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:41:56.566488 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.566360    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q22bk\" (UniqueName: \"kubernetes.io/projected/bb5e9022-8d73-46cc-9a8a-1c026c307c69-kube-api-access-q22bk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r\" (UID: \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:41:56.566488 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.566431    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb5e9022-8d73-46cc-9a8a-1c026c307c69-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r\" (UID: \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:41:56.667622 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.667587    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb5e9022-8d73-46cc-9a8a-1c026c307c69-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r\" (UID: \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:41:56.667830 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.667640    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb5e9022-8d73-46cc-9a8a-1c026c307c69-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r\" (UID: \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:41:56.667830 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.667721    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q22bk\" (UniqueName: \"kubernetes.io/projected/bb5e9022-8d73-46cc-9a8a-1c026c307c69-kube-api-access-q22bk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r\" (UID: \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:41:56.668031 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.668009    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb5e9022-8d73-46cc-9a8a-1c026c307c69-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r\" (UID: \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:41:56.668102 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.668050    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb5e9022-8d73-46cc-9a8a-1c026c307c69-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r\" (UID: \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:41:56.675509 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.675487    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q22bk\" (UniqueName: \"kubernetes.io/projected/bb5e9022-8d73-46cc-9a8a-1c026c307c69-kube-api-access-q22bk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r\" (UID: \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:41:56.762122 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.762048    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:41:56.894247 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:56.894221    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"]
Jul 10 03:41:56.896300 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:41:56.896274    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5e9022_8d73_46cc_9a8a_1c026c307c69.slice/crio-22ec7e233cb5a7d623a4859a0aa57eded5e8dbbe2d4a660d62cfd2c5c3380d16 WatchSource:0}: Error finding container 22ec7e233cb5a7d623a4859a0aa57eded5e8dbbe2d4a660d62cfd2c5c3380d16: Status 404 returned error can't find the container with id 22ec7e233cb5a7d623a4859a0aa57eded5e8dbbe2d4a660d62cfd2c5c3380d16
Jul 10 03:41:57.246588 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.246556    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"]
Jul 10 03:41:57.250001 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.249984    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:41:57.258800 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.258771    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"]
Jul 10 03:41:57.298868 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.298830    2561 generic.go:356] "Generic (PLEG): container finished" podID="bb5e9022-8d73-46cc-9a8a-1c026c307c69" containerID="b42ade59b99467c90ffb3c602637a468a7b026118d1d54e30bd0d62551ebf570" exitCode=0
Jul 10 03:41:57.299034 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.298925    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r" event={"ID":"bb5e9022-8d73-46cc-9a8a-1c026c307c69","Type":"ContainerDied","Data":"b42ade59b99467c90ffb3c602637a468a7b026118d1d54e30bd0d62551ebf570"}
Jul 10 03:41:57.299034 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.299006    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r" event={"ID":"bb5e9022-8d73-46cc-9a8a-1c026c307c69","Type":"ContainerStarted","Data":"22ec7e233cb5a7d623a4859a0aa57eded5e8dbbe2d4a660d62cfd2c5c3380d16"}
Jul 10 03:41:57.373872 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.373836    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed03f018-31e5-42e3-9ed0-f676d890a384-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq\" (UID: \"ed03f018-31e5-42e3-9ed0-f676d890a384\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:41:57.374037 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.373947    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph5ct\" (UniqueName: \"kubernetes.io/projected/ed03f018-31e5-42e3-9ed0-f676d890a384-kube-api-access-ph5ct\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq\" (UID: \"ed03f018-31e5-42e3-9ed0-f676d890a384\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:41:57.374037 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.373974    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed03f018-31e5-42e3-9ed0-f676d890a384-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq\" (UID: \"ed03f018-31e5-42e3-9ed0-f676d890a384\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:41:57.474411 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.474377    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed03f018-31e5-42e3-9ed0-f676d890a384-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq\" (UID: \"ed03f018-31e5-42e3-9ed0-f676d890a384\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:41:57.474924 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.474446    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph5ct\" (UniqueName: \"kubernetes.io/projected/ed03f018-31e5-42e3-9ed0-f676d890a384-kube-api-access-ph5ct\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq\" (UID: \"ed03f018-31e5-42e3-9ed0-f676d890a384\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:41:57.474924 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.474470    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed03f018-31e5-42e3-9ed0-f676d890a384-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq\" (UID: \"ed03f018-31e5-42e3-9ed0-f676d890a384\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:41:57.474924 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.474871    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed03f018-31e5-42e3-9ed0-f676d890a384-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq\" (UID: \"ed03f018-31e5-42e3-9ed0-f676d890a384\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:41:57.475043 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.474922    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed03f018-31e5-42e3-9ed0-f676d890a384-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq\" (UID: \"ed03f018-31e5-42e3-9ed0-f676d890a384\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:41:57.482761 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.482743    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph5ct\" (UniqueName: \"kubernetes.io/projected/ed03f018-31e5-42e3-9ed0-f676d890a384-kube-api-access-ph5ct\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq\" (UID: \"ed03f018-31e5-42e3-9ed0-f676d890a384\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:41:57.559761 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.559691    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:41:57.691057 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.691031    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"]
Jul 10 03:41:57.692948 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:41:57.692916    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded03f018_31e5_42e3_9ed0_f676d890a384.slice/crio-46d9a0f40eb9fd436001e65d498a0925ccb0e033126a2c5dbd6e8ef13da8a3c5 WatchSource:0}: Error finding container 46d9a0f40eb9fd436001e65d498a0925ccb0e033126a2c5dbd6e8ef13da8a3c5: Status 404 returned error can't find the container with id 46d9a0f40eb9fd436001e65d498a0925ccb0e033126a2c5dbd6e8ef13da8a3c5
Jul 10 03:41:57.847750 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.847718    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"]
Jul 10 03:41:57.851317 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.851301    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:41:57.860689 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.860662    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"]
Jul 10 03:41:57.978452 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.978416    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq\" (UID: \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:41:57.978604 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.978459    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq\" (UID: \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:41:57.978604 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:57.978533    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk7gs\" (UniqueName: \"kubernetes.io/projected/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-kube-api-access-fk7gs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq\" (UID: \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:41:58.079140 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.079105    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk7gs\" (UniqueName: \"kubernetes.io/projected/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-kube-api-access-fk7gs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq\" (UID: \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:41:58.079298 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.079187    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq\" (UID: \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:41:58.079298 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.079231    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq\" (UID: \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:41:58.079579 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.079558    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq\" (UID: \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:41:58.079616 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.079590    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq\" (UID: \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:41:58.086703 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.086641    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk7gs\" (UniqueName: \"kubernetes.io/projected/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-kube-api-access-fk7gs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq\" (UID: \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:41:58.161249 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.161177    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:41:58.246162 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.246118    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"]
Jul 10 03:41:58.250912 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.250886    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:41:58.257962 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.257937    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"]
Jul 10 03:41:58.296105 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.296083    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"]
Jul 10 03:41:58.298553 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:41:58.298527    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5fedb8f_fbc2_4244_921c_5b12e9790b0c.slice/crio-7a2775e47a5f7850b77d9008e1c351052f371c278b44101921c0cee56f559298 WatchSource:0}: Error finding container 7a2775e47a5f7850b77d9008e1c351052f371c278b44101921c0cee56f559298: Status 404 returned error can't find the container with id 7a2775e47a5f7850b77d9008e1c351052f371c278b44101921c0cee56f559298
Jul 10 03:41:58.304847 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.304824    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq" event={"ID":"d5fedb8f-fbc2-4244-921c-5b12e9790b0c","Type":"ContainerStarted","Data":"7a2775e47a5f7850b77d9008e1c351052f371c278b44101921c0cee56f559298"}
Jul 10 03:41:58.306302 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.306266    2561 generic.go:356] "Generic (PLEG): container finished" podID="ed03f018-31e5-42e3-9ed0-f676d890a384" containerID="fb71b9e3201016bde0e31da016554a97de2788a5baf872b1fa43c05fc2c1a79f" exitCode=0
Jul 10 03:41:58.306396 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.306343    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq" event={"ID":"ed03f018-31e5-42e3-9ed0-f676d890a384","Type":"ContainerDied","Data":"fb71b9e3201016bde0e31da016554a97de2788a5baf872b1fa43c05fc2c1a79f"}
Jul 10 03:41:58.306396 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.306391    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq" event={"ID":"ed03f018-31e5-42e3-9ed0-f676d890a384","Type":"ContainerStarted","Data":"46d9a0f40eb9fd436001e65d498a0925ccb0e033126a2c5dbd6e8ef13da8a3c5"}
Jul 10 03:41:58.308286 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.308265    2561 generic.go:356] "Generic (PLEG): container finished" podID="bb5e9022-8d73-46cc-9a8a-1c026c307c69" containerID="8a68af5fe4775e619a646bb1295c98ee8b4ae89de4973cbf4367fc40c28ae13a" exitCode=0
Jul 10 03:41:58.308385 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.308294    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r" event={"ID":"bb5e9022-8d73-46cc-9a8a-1c026c307c69","Type":"ContainerDied","Data":"8a68af5fe4775e619a646bb1295c98ee8b4ae89de4973cbf4367fc40c28ae13a"}
Jul 10 03:41:58.381955 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.381922    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c78c13-e85e-4a75-a4f8-d351da005359-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4\" (UID: \"87c78c13-e85e-4a75-a4f8-d351da005359\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:41:58.382095 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.381964    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c78c13-e85e-4a75-a4f8-d351da005359-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4\" (UID: \"87c78c13-e85e-4a75-a4f8-d351da005359\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:41:58.382207 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.382135    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkn8c\" (UniqueName: \"kubernetes.io/projected/87c78c13-e85e-4a75-a4f8-d351da005359-kube-api-access-nkn8c\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4\" (UID: \"87c78c13-e85e-4a75-a4f8-d351da005359\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:41:58.483064 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.483029    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkn8c\" (UniqueName: \"kubernetes.io/projected/87c78c13-e85e-4a75-a4f8-d351da005359-kube-api-access-nkn8c\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4\" (UID: \"87c78c13-e85e-4a75-a4f8-d351da005359\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:41:58.483446 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.483107    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c78c13-e85e-4a75-a4f8-d351da005359-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4\" (UID: \"87c78c13-e85e-4a75-a4f8-d351da005359\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:41:58.483446 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.483140    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c78c13-e85e-4a75-a4f8-d351da005359-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4\" (UID: \"87c78c13-e85e-4a75-a4f8-d351da005359\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:41:58.483555 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.483526    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c78c13-e85e-4a75-a4f8-d351da005359-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4\" (UID: \"87c78c13-e85e-4a75-a4f8-d351da005359\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:41:58.483611 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.483563    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c78c13-e85e-4a75-a4f8-d351da005359-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4\" (UID: \"87c78c13-e85e-4a75-a4f8-d351da005359\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:41:58.491502 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.491480    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkn8c\" (UniqueName: \"kubernetes.io/projected/87c78c13-e85e-4a75-a4f8-d351da005359-kube-api-access-nkn8c\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4\" (UID: \"87c78c13-e85e-4a75-a4f8-d351da005359\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:41:58.562519 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.562487    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:41:58.692684 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:58.692594    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"]
Jul 10 03:41:58.694926 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:41:58.694887    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c78c13_e85e_4a75_a4f8_d351da005359.slice/crio-eae691f736d0300f6a4980e275dbd1fa520f9c2a50b8e25c12d0d687f529a005 WatchSource:0}: Error finding container eae691f736d0300f6a4980e275dbd1fa520f9c2a50b8e25c12d0d687f529a005: Status 404 returned error can't find the container with id eae691f736d0300f6a4980e275dbd1fa520f9c2a50b8e25c12d0d687f529a005
Jul 10 03:41:59.315099 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:59.315069    2561 generic.go:356] "Generic (PLEG): container finished" podID="bb5e9022-8d73-46cc-9a8a-1c026c307c69" containerID="c31d2bd9548772db9171444922baaea97b649e80a0c45f231eb743d5dd5690f3" exitCode=0
Jul 10 03:41:59.315260 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:59.315164    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r" event={"ID":"bb5e9022-8d73-46cc-9a8a-1c026c307c69","Type":"ContainerDied","Data":"c31d2bd9548772db9171444922baaea97b649e80a0c45f231eb743d5dd5690f3"}
Jul 10 03:41:59.316612 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:59.316592    2561 generic.go:356] "Generic (PLEG): container finished" podID="87c78c13-e85e-4a75-a4f8-d351da005359" containerID="1aa0e06653e8eaa2e432738848f0c0107447789915511d2ec8923624567a9daa" exitCode=0
Jul 10 03:41:59.316749 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:59.316618    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4" event={"ID":"87c78c13-e85e-4a75-a4f8-d351da005359","Type":"ContainerDied","Data":"1aa0e06653e8eaa2e432738848f0c0107447789915511d2ec8923624567a9daa"}
Jul 10 03:41:59.316749 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:59.316681    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4" event={"ID":"87c78c13-e85e-4a75-a4f8-d351da005359","Type":"ContainerStarted","Data":"eae691f736d0300f6a4980e275dbd1fa520f9c2a50b8e25c12d0d687f529a005"}
Jul 10 03:41:59.318356 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:59.318332    2561 generic.go:356] "Generic (PLEG): container finished" podID="d5fedb8f-fbc2-4244-921c-5b12e9790b0c" containerID="d8088caab2b52f8684f81698125e8e38a1d9432a6dea594cfe485cf08b2aa37f" exitCode=0
Jul 10 03:41:59.318461 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:59.318405    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq" event={"ID":"d5fedb8f-fbc2-4244-921c-5b12e9790b0c","Type":"ContainerDied","Data":"d8088caab2b52f8684f81698125e8e38a1d9432a6dea594cfe485cf08b2aa37f"}
Jul 10 03:41:59.320361 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:59.320342    2561 generic.go:356] "Generic (PLEG): container finished" podID="ed03f018-31e5-42e3-9ed0-f676d890a384" containerID="d2f0250861847a51071aa9ad3bc8d030bdb2a513ca6414e3783f4e9e3aafd1a5" exitCode=0
Jul 10 03:41:59.320471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:41:59.320426    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq" event={"ID":"ed03f018-31e5-42e3-9ed0-f676d890a384","Type":"ContainerDied","Data":"d2f0250861847a51071aa9ad3bc8d030bdb2a513ca6414e3783f4e9e3aafd1a5"}
Jul 10 03:42:00.326701 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.326672    2561 generic.go:356] "Generic (PLEG): container finished" podID="d5fedb8f-fbc2-4244-921c-5b12e9790b0c" containerID="34eb8198b5f498a236214f800ade3f966effc1f32f56d3967624fd99101cf196" exitCode=0
Jul 10 03:42:00.327117 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.326767    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq" event={"ID":"d5fedb8f-fbc2-4244-921c-5b12e9790b0c","Type":"ContainerDied","Data":"34eb8198b5f498a236214f800ade3f966effc1f32f56d3967624fd99101cf196"}
Jul 10 03:42:00.329009 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.328986    2561 generic.go:356] "Generic (PLEG): container finished" podID="ed03f018-31e5-42e3-9ed0-f676d890a384" containerID="1b51a701e0835e45de9cfefa0b02cced7d3e7d516cf98a7226b790649a3bc72a" exitCode=0
Jul 10 03:42:00.329107 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.329056    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq" event={"ID":"ed03f018-31e5-42e3-9ed0-f676d890a384","Type":"ContainerDied","Data":"1b51a701e0835e45de9cfefa0b02cced7d3e7d516cf98a7226b790649a3bc72a"}
Jul 10 03:42:00.330992 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.330974    2561 generic.go:356] "Generic (PLEG): container finished" podID="87c78c13-e85e-4a75-a4f8-d351da005359" containerID="335f869b785f8a94044785e3ea73e2f1e3376a22026cd0973a4efdb076d6aca5" exitCode=0
Jul 10 03:42:00.331059 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.331034    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4" event={"ID":"87c78c13-e85e-4a75-a4f8-d351da005359","Type":"ContainerDied","Data":"335f869b785f8a94044785e3ea73e2f1e3376a22026cd0973a4efdb076d6aca5"}
Jul 10 03:42:00.468120 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.468092    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:42:00.601829 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.601749    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb5e9022-8d73-46cc-9a8a-1c026c307c69-util\") pod \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\" (UID: \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\") "
Jul 10 03:42:00.601829 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.601811    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb5e9022-8d73-46cc-9a8a-1c026c307c69-bundle\") pod \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\" (UID: \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\") "
Jul 10 03:42:00.601982 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.601871    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q22bk\" (UniqueName: \"kubernetes.io/projected/bb5e9022-8d73-46cc-9a8a-1c026c307c69-kube-api-access-q22bk\") pod \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\" (UID: \"bb5e9022-8d73-46cc-9a8a-1c026c307c69\") "
Jul 10 03:42:00.602450 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.602423    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb5e9022-8d73-46cc-9a8a-1c026c307c69-bundle" (OuterVolumeSpecName: "bundle") pod "bb5e9022-8d73-46cc-9a8a-1c026c307c69" (UID: "bb5e9022-8d73-46cc-9a8a-1c026c307c69"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:42:00.604067 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.604042    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb5e9022-8d73-46cc-9a8a-1c026c307c69-kube-api-access-q22bk" (OuterVolumeSpecName: "kube-api-access-q22bk") pod "bb5e9022-8d73-46cc-9a8a-1c026c307c69" (UID: "bb5e9022-8d73-46cc-9a8a-1c026c307c69"). InnerVolumeSpecName "kube-api-access-q22bk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:42:00.606697 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.606643    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb5e9022-8d73-46cc-9a8a-1c026c307c69-util" (OuterVolumeSpecName: "util") pod "bb5e9022-8d73-46cc-9a8a-1c026c307c69" (UID: "bb5e9022-8d73-46cc-9a8a-1c026c307c69"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:42:00.703246 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.703213    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q22bk\" (UniqueName: \"kubernetes.io/projected/bb5e9022-8d73-46cc-9a8a-1c026c307c69-kube-api-access-q22bk\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:42:00.703246 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.703240    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb5e9022-8d73-46cc-9a8a-1c026c307c69-util\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:42:00.703246 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:00.703251    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb5e9022-8d73-46cc-9a8a-1c026c307c69-bundle\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:42:01.338730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.338698    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r"
Jul 10 03:42:01.338730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.338711    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r" event={"ID":"bb5e9022-8d73-46cc-9a8a-1c026c307c69","Type":"ContainerDied","Data":"22ec7e233cb5a7d623a4859a0aa57eded5e8dbbe2d4a660d62cfd2c5c3380d16"}
Jul 10 03:42:01.339218 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.338767    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22ec7e233cb5a7d623a4859a0aa57eded5e8dbbe2d4a660d62cfd2c5c3380d16"
Jul 10 03:42:01.341054 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.341028    2561 generic.go:356] "Generic (PLEG): container finished" podID="87c78c13-e85e-4a75-a4f8-d351da005359" containerID="4ed66cd9a2884994fef0d9c55aa1365dde5019dbc636ea57bd849ea4bda3a7f5" exitCode=0
Jul 10 03:42:01.341168 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.341116    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4" event={"ID":"87c78c13-e85e-4a75-a4f8-d351da005359","Type":"ContainerDied","Data":"4ed66cd9a2884994fef0d9c55aa1365dde5019dbc636ea57bd849ea4bda3a7f5"}
Jul 10 03:42:01.343185 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.343166    2561 generic.go:356] "Generic (PLEG): container finished" podID="d5fedb8f-fbc2-4244-921c-5b12e9790b0c" containerID="a7df0f721dbe84ba50d5f8c9f10bfaf86fcc9fd5c15398bb0068678227b3c677" exitCode=0
Jul 10 03:42:01.343290 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.343248    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq" event={"ID":"d5fedb8f-fbc2-4244-921c-5b12e9790b0c","Type":"ContainerDied","Data":"a7df0f721dbe84ba50d5f8c9f10bfaf86fcc9fd5c15398bb0068678227b3c677"}
Jul 10 03:42:01.479351 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.479328    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:42:01.612074 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.611989    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed03f018-31e5-42e3-9ed0-f676d890a384-bundle\") pod \"ed03f018-31e5-42e3-9ed0-f676d890a384\" (UID: \"ed03f018-31e5-42e3-9ed0-f676d890a384\") "
Jul 10 03:42:01.612074 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.612042    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed03f018-31e5-42e3-9ed0-f676d890a384-util\") pod \"ed03f018-31e5-42e3-9ed0-f676d890a384\" (UID: \"ed03f018-31e5-42e3-9ed0-f676d890a384\") "
Jul 10 03:42:01.612291 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.612120    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ph5ct\" (UniqueName: \"kubernetes.io/projected/ed03f018-31e5-42e3-9ed0-f676d890a384-kube-api-access-ph5ct\") pod \"ed03f018-31e5-42e3-9ed0-f676d890a384\" (UID: \"ed03f018-31e5-42e3-9ed0-f676d890a384\") "
Jul 10 03:42:01.612459 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.612428    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed03f018-31e5-42e3-9ed0-f676d890a384-bundle" (OuterVolumeSpecName: "bundle") pod "ed03f018-31e5-42e3-9ed0-f676d890a384" (UID: "ed03f018-31e5-42e3-9ed0-f676d890a384"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:42:01.614222 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.614181    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed03f018-31e5-42e3-9ed0-f676d890a384-kube-api-access-ph5ct" (OuterVolumeSpecName: "kube-api-access-ph5ct") pod "ed03f018-31e5-42e3-9ed0-f676d890a384" (UID: "ed03f018-31e5-42e3-9ed0-f676d890a384"). InnerVolumeSpecName "kube-api-access-ph5ct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:42:01.617089 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.617057    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed03f018-31e5-42e3-9ed0-f676d890a384-util" (OuterVolumeSpecName: "util") pod "ed03f018-31e5-42e3-9ed0-f676d890a384" (UID: "ed03f018-31e5-42e3-9ed0-f676d890a384"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:42:01.712746 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.712702    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph5ct\" (UniqueName: \"kubernetes.io/projected/ed03f018-31e5-42e3-9ed0-f676d890a384-kube-api-access-ph5ct\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:42:01.712746 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.712736    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed03f018-31e5-42e3-9ed0-f676d890a384-bundle\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:42:01.712746 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:01.712745    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed03f018-31e5-42e3-9ed0-f676d890a384-util\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:42:02.350575 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.350546    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq"
Jul 10 03:42:02.351018 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.350576    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq" event={"ID":"ed03f018-31e5-42e3-9ed0-f676d890a384","Type":"ContainerDied","Data":"46d9a0f40eb9fd436001e65d498a0925ccb0e033126a2c5dbd6e8ef13da8a3c5"}
Jul 10 03:42:02.351018 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.350627    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46d9a0f40eb9fd436001e65d498a0925ccb0e033126a2c5dbd6e8ef13da8a3c5"
Jul 10 03:42:02.521776 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.521753    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:42:02.523826 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.523808    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:42:02.621206 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.621129    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-util\") pod \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\" (UID: \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\") "
Jul 10 03:42:02.621206 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.621164    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c78c13-e85e-4a75-a4f8-d351da005359-bundle\") pod \"87c78c13-e85e-4a75-a4f8-d351da005359\" (UID: \"87c78c13-e85e-4a75-a4f8-d351da005359\") "
Jul 10 03:42:02.621206 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.621196    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-bundle\") pod \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\" (UID: \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\") "
Jul 10 03:42:02.621477 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.621232    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk7gs\" (UniqueName: \"kubernetes.io/projected/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-kube-api-access-fk7gs\") pod \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\" (UID: \"d5fedb8f-fbc2-4244-921c-5b12e9790b0c\") "
Jul 10 03:42:02.621477 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.621267    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c78c13-e85e-4a75-a4f8-d351da005359-util\") pod \"87c78c13-e85e-4a75-a4f8-d351da005359\" (UID: \"87c78c13-e85e-4a75-a4f8-d351da005359\") "
Jul 10 03:42:02.621477 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.621287    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkn8c\" (UniqueName: \"kubernetes.io/projected/87c78c13-e85e-4a75-a4f8-d351da005359-kube-api-access-nkn8c\") pod \"87c78c13-e85e-4a75-a4f8-d351da005359\" (UID: \"87c78c13-e85e-4a75-a4f8-d351da005359\") "
Jul 10 03:42:02.621851 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.621797    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-bundle" (OuterVolumeSpecName: "bundle") pod "d5fedb8f-fbc2-4244-921c-5b12e9790b0c" (UID: "d5fedb8f-fbc2-4244-921c-5b12e9790b0c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:42:02.621973 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.621916    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c78c13-e85e-4a75-a4f8-d351da005359-bundle" (OuterVolumeSpecName: "bundle") pod "87c78c13-e85e-4a75-a4f8-d351da005359" (UID: "87c78c13-e85e-4a75-a4f8-d351da005359"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:42:02.623682 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.623660    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-kube-api-access-fk7gs" (OuterVolumeSpecName: "kube-api-access-fk7gs") pod "d5fedb8f-fbc2-4244-921c-5b12e9790b0c" (UID: "d5fedb8f-fbc2-4244-921c-5b12e9790b0c"). InnerVolumeSpecName "kube-api-access-fk7gs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:42:02.623902 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.623871    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87c78c13-e85e-4a75-a4f8-d351da005359-kube-api-access-nkn8c" (OuterVolumeSpecName: "kube-api-access-nkn8c") pod "87c78c13-e85e-4a75-a4f8-d351da005359" (UID: "87c78c13-e85e-4a75-a4f8-d351da005359"). InnerVolumeSpecName "kube-api-access-nkn8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:42:02.627142 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.627115    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-util" (OuterVolumeSpecName: "util") pod "d5fedb8f-fbc2-4244-921c-5b12e9790b0c" (UID: "d5fedb8f-fbc2-4244-921c-5b12e9790b0c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:42:02.627886 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.627863    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c78c13-e85e-4a75-a4f8-d351da005359-util" (OuterVolumeSpecName: "util") pod "87c78c13-e85e-4a75-a4f8-d351da005359" (UID: "87c78c13-e85e-4a75-a4f8-d351da005359"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:42:02.727566 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.722810    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c78c13-e85e-4a75-a4f8-d351da005359-util\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:42:02.727566 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.722859    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkn8c\" (UniqueName: \"kubernetes.io/projected/87c78c13-e85e-4a75-a4f8-d351da005359-kube-api-access-nkn8c\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:42:02.727566 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.722874    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-util\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:42:02.727566 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.722888    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c78c13-e85e-4a75-a4f8-d351da005359-bundle\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:42:02.727566 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.722901    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-bundle\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:42:02.727566 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:02.722919    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk7gs\" (UniqueName: \"kubernetes.io/projected/d5fedb8f-fbc2-4244-921c-5b12e9790b0c-kube-api-access-fk7gs\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:42:03.357515 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:03.357485    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4"
Jul 10 03:42:03.357515 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:03.357492    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4" event={"ID":"87c78c13-e85e-4a75-a4f8-d351da005359","Type":"ContainerDied","Data":"eae691f736d0300f6a4980e275dbd1fa520f9c2a50b8e25c12d0d687f529a005"}
Jul 10 03:42:03.357990 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:03.357537    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eae691f736d0300f6a4980e275dbd1fa520f9c2a50b8e25c12d0d687f529a005"
Jul 10 03:42:03.359642 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:03.359583    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq"
Jul 10 03:42:03.359642 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:03.359608    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq" event={"ID":"d5fedb8f-fbc2-4244-921c-5b12e9790b0c","Type":"ContainerDied","Data":"7a2775e47a5f7850b77d9008e1c351052f371c278b44101921c0cee56f559298"}
Jul 10 03:42:03.359828 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:03.359670    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a2775e47a5f7850b77d9008e1c351052f371c278b44101921c0cee56f559298"
Jul 10 03:42:10.208432 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:10.208389    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:13.815829 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:13.815790    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:16.848016 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:16.847974    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:18.283556 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:18.283516    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:18.985440 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:18.985414    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:42:18.986047 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:18.986025    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:42:19.210205 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:19.210169    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:21.414193 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:21.414154    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:21.832971 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:21.832927    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:22.051698 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:22.051668    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:24.847252 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:24.847213    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:25.045230 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045197    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fd8999687-gn9xg"]
Jul 10 03:42:25.045668 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045629    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c78c13-e85e-4a75-a4f8-d351da005359" containerName="pull"
Jul 10 03:42:25.045668 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045664    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c78c13-e85e-4a75-a4f8-d351da005359" containerName="pull"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045682    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed03f018-31e5-42e3-9ed0-f676d890a384" containerName="pull"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045690    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed03f018-31e5-42e3-9ed0-f676d890a384" containerName="pull"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045704    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c78c13-e85e-4a75-a4f8-d351da005359" containerName="extract"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045712    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c78c13-e85e-4a75-a4f8-d351da005359" containerName="extract"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045723    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5fedb8f-fbc2-4244-921c-5b12e9790b0c" containerName="util"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045732    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5fedb8f-fbc2-4244-921c-5b12e9790b0c" containerName="util"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045741    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5e9022-8d73-46cc-9a8a-1c026c307c69" containerName="util"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045749    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5e9022-8d73-46cc-9a8a-1c026c307c69" containerName="util"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045766    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5fedb8f-fbc2-4244-921c-5b12e9790b0c" containerName="extract"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045776    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5fedb8f-fbc2-4244-921c-5b12e9790b0c" containerName="extract"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045794    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c78c13-e85e-4a75-a4f8-d351da005359" containerName="util"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045803    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c78c13-e85e-4a75-a4f8-d351da005359" containerName="util"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045815    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5e9022-8d73-46cc-9a8a-1c026c307c69" containerName="pull"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045823    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5e9022-8d73-46cc-9a8a-1c026c307c69" containerName="pull"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045831    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed03f018-31e5-42e3-9ed0-f676d890a384" containerName="extract"
Jul 10 03:42:25.045839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045839    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed03f018-31e5-42e3-9ed0-f676d890a384" containerName="extract"
Jul 10 03:42:25.046742 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045849    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed03f018-31e5-42e3-9ed0-f676d890a384" containerName="util"
Jul 10 03:42:25.046742 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045857    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed03f018-31e5-42e3-9ed0-f676d890a384" containerName="util"
Jul 10 03:42:25.046742 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045866    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5e9022-8d73-46cc-9a8a-1c026c307c69" containerName="extract"
Jul 10 03:42:25.046742 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045874    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5e9022-8d73-46cc-9a8a-1c026c307c69" containerName="extract"
Jul 10 03:42:25.046742 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045882    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5fedb8f-fbc2-4244-921c-5b12e9790b0c" containerName="pull"
Jul 10 03:42:25.046742 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045890    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5fedb8f-fbc2-4244-921c-5b12e9790b0c" containerName="pull"
Jul 10 03:42:25.046742 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045983    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5fedb8f-fbc2-4244-921c-5b12e9790b0c" containerName="extract"
Jul 10 03:42:25.046742 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.045996    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed03f018-31e5-42e3-9ed0-f676d890a384" containerName="extract"
Jul 10 03:42:25.046742 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.046006    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb5e9022-8d73-46cc-9a8a-1c026c307c69" containerName="extract"
Jul 10 03:42:25.046742 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.046020    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="87c78c13-e85e-4a75-a4f8-d351da005359" containerName="extract"
Jul 10 03:42:25.049378 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.049348    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.067726 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.067693    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fd8999687-gn9xg"]
Jul 10 03:42:25.094792 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.094757    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d825549b-7e39-46ee-a09f-d9323079ee6b-console-config\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.094938 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.094798    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d825549b-7e39-46ee-a09f-d9323079ee6b-console-oauth-config\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.094938 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.094877    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ngxg\" (UniqueName: \"kubernetes.io/projected/d825549b-7e39-46ee-a09f-d9323079ee6b-kube-api-access-4ngxg\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.094938 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.094920    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d825549b-7e39-46ee-a09f-d9323079ee6b-oauth-serving-cert\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.095046 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.094942    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d825549b-7e39-46ee-a09f-d9323079ee6b-trusted-ca-bundle\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.095046 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.094960    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d825549b-7e39-46ee-a09f-d9323079ee6b-console-serving-cert\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.095046 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.095027    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d825549b-7e39-46ee-a09f-d9323079ee6b-service-ca\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.196130 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.196031    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d825549b-7e39-46ee-a09f-d9323079ee6b-oauth-serving-cert\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.196130 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.196080    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d825549b-7e39-46ee-a09f-d9323079ee6b-trusted-ca-bundle\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.196130 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.196106    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d825549b-7e39-46ee-a09f-d9323079ee6b-console-serving-cert\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.196405 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.196154    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d825549b-7e39-46ee-a09f-d9323079ee6b-service-ca\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.196405 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.196183    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d825549b-7e39-46ee-a09f-d9323079ee6b-console-config\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.196405 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.196223    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d825549b-7e39-46ee-a09f-d9323079ee6b-console-oauth-config\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.196405 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.196283    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ngxg\" (UniqueName: \"kubernetes.io/projected/d825549b-7e39-46ee-a09f-d9323079ee6b-kube-api-access-4ngxg\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.197022 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.196985    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d825549b-7e39-46ee-a09f-d9323079ee6b-oauth-serving-cert\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.197022 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.197015    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d825549b-7e39-46ee-a09f-d9323079ee6b-trusted-ca-bundle\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.197232 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.196994    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d825549b-7e39-46ee-a09f-d9323079ee6b-console-config\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.197301 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.197282    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d825549b-7e39-46ee-a09f-d9323079ee6b-service-ca\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.198858 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.198833    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d825549b-7e39-46ee-a09f-d9323079ee6b-console-serving-cert\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.198959 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.198836    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d825549b-7e39-46ee-a09f-d9323079ee6b-console-oauth-config\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.208690 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.208664    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ngxg\" (UniqueName: \"kubernetes.io/projected/d825549b-7e39-46ee-a09f-d9323079ee6b-kube-api-access-4ngxg\") pod \"console-7fd8999687-gn9xg\" (UID: \"d825549b-7e39-46ee-a09f-d9323079ee6b\") " pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.360772 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.360730    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:25.504238 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.504213    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fd8999687-gn9xg"]
Jul 10 03:42:25.506102 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:42:25.506077    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd825549b_7e39_46ee_a09f_d9323079ee6b.slice/crio-37c9b4bd6ea6296c2760df48686b117c52785e804f85cbab5d9534364039ec90 WatchSource:0}: Error finding container 37c9b4bd6ea6296c2760df48686b117c52785e804f85cbab5d9534364039ec90: Status 404 returned error can't find the container with id 37c9b4bd6ea6296c2760df48686b117c52785e804f85cbab5d9534364039ec90
Jul 10 03:42:25.988632 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:25.988602    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:26.230671 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:26.230620    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:26.326854 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:26.326821    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:26.477485 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:26.477450    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fd8999687-gn9xg" event={"ID":"d825549b-7e39-46ee-a09f-d9323079ee6b","Type":"ContainerStarted","Data":"f8f94ba689a3d2483ebc3c5df500f2d9fba3cfd2dd68b967a2361a1d5a25e089"}
Jul 10 03:42:26.477694 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:26.477502    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fd8999687-gn9xg" event={"ID":"d825549b-7e39-46ee-a09f-d9323079ee6b","Type":"ContainerStarted","Data":"37c9b4bd6ea6296c2760df48686b117c52785e804f85cbab5d9534364039ec90"}
Jul 10 03:42:26.498184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:26.498142    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fd8999687-gn9xg" podStartSLOduration=1.498129027 podStartE2EDuration="1.498129027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:42:26.497983171 +0000 UTC m=+607.988979081" watchObservedRunningTime="2026-07-10 03:42:26.498129027 +0000 UTC m=+607.989124899"
Jul 10 03:42:27.471440 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.471405    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"]
Jul 10 03:42:27.477050 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.477022    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"
Jul 10 03:42:27.479501 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.479475    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 10 03:42:27.479677 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.479522    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 03:42:27.479677 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.479487    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 03:42:27.481462 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.481442    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8p5ls\""
Jul 10 03:42:27.481807 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.481456    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 10 03:42:27.490840 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.490818    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"]
Jul 10 03:42:27.491761 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.491737    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:27.519066 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.519032    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g6gg\" (UniqueName: \"kubernetes.io/projected/197c9c66-7a0e-4317-a816-b6db7422d5d6-kube-api-access-7g6gg\") pod \"kuadrant-console-plugin-66b5774985-wbcrq\" (UID: \"197c9c66-7a0e-4317-a816-b6db7422d5d6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"
Jul 10 03:42:27.519209 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.519102    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/197c9c66-7a0e-4317-a816-b6db7422d5d6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wbcrq\" (UID: \"197c9c66-7a0e-4317-a816-b6db7422d5d6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"
Jul 10 03:42:27.519367 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.519342    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/197c9c66-7a0e-4317-a816-b6db7422d5d6-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wbcrq\" (UID: \"197c9c66-7a0e-4317-a816-b6db7422d5d6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"
Jul 10 03:42:27.620523 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.620487    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g6gg\" (UniqueName: \"kubernetes.io/projected/197c9c66-7a0e-4317-a816-b6db7422d5d6-kube-api-access-7g6gg\") pod \"kuadrant-console-plugin-66b5774985-wbcrq\" (UID: \"197c9c66-7a0e-4317-a816-b6db7422d5d6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"
Jul 10 03:42:27.620806 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.620542    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/197c9c66-7a0e-4317-a816-b6db7422d5d6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wbcrq\" (UID: \"197c9c66-7a0e-4317-a816-b6db7422d5d6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"
Jul 10 03:42:27.620806 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.620590    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/197c9c66-7a0e-4317-a816-b6db7422d5d6-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wbcrq\" (UID: \"197c9c66-7a0e-4317-a816-b6db7422d5d6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"
Jul 10 03:42:27.620806 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:42:27.620728    2561 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 10 03:42:27.620991 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:42:27.620820    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197c9c66-7a0e-4317-a816-b6db7422d5d6-plugin-serving-cert podName:197c9c66-7a0e-4317-a816-b6db7422d5d6 nodeName:}" failed. No retries permitted until 2026-07-10 03:42:28.12079842 +0000 UTC m=+609.611794274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/197c9c66-7a0e-4317-a816-b6db7422d5d6-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-wbcrq" (UID: "197c9c66-7a0e-4317-a816-b6db7422d5d6") : secret "plugin-serving-cert" not found
Jul 10 03:42:27.621208 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.621189    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/197c9c66-7a0e-4317-a816-b6db7422d5d6-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wbcrq\" (UID: \"197c9c66-7a0e-4317-a816-b6db7422d5d6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"
Jul 10 03:42:27.634000 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.633975    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g6gg\" (UniqueName: \"kubernetes.io/projected/197c9c66-7a0e-4317-a816-b6db7422d5d6-kube-api-access-7g6gg\") pod \"kuadrant-console-plugin-66b5774985-wbcrq\" (UID: \"197c9c66-7a0e-4317-a816-b6db7422d5d6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"
Jul 10 03:42:27.736137 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:27.736051    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:28.124792 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:28.124752    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/197c9c66-7a0e-4317-a816-b6db7422d5d6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wbcrq\" (UID: \"197c9c66-7a0e-4317-a816-b6db7422d5d6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"
Jul 10 03:42:28.127225 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:28.127191    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/197c9c66-7a0e-4317-a816-b6db7422d5d6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wbcrq\" (UID: \"197c9c66-7a0e-4317-a816-b6db7422d5d6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"
Jul 10 03:42:28.395300 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:28.395203    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"
Jul 10 03:42:28.533025 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:28.533000    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq"]
Jul 10 03:42:28.533942 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:42:28.533917    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod197c9c66_7a0e_4317_a816_b6db7422d5d6.slice/crio-314dbddeb13f4ad58464a4cc27fbabcfb9f1dbf27af25e087ac1922c0a9a1c67 WatchSource:0}: Error finding container 314dbddeb13f4ad58464a4cc27fbabcfb9f1dbf27af25e087ac1922c0a9a1c67: Status 404 returned error can't find the container with id 314dbddeb13f4ad58464a4cc27fbabcfb9f1dbf27af25e087ac1922c0a9a1c67
Jul 10 03:42:29.505602 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:29.505566    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq" event={"ID":"197c9c66-7a0e-4317-a816-b6db7422d5d6","Type":"ContainerStarted","Data":"314dbddeb13f4ad58464a4cc27fbabcfb9f1dbf27af25e087ac1922c0a9a1c67"}
Jul 10 03:42:29.812010 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:29.811927    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:29.912456 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:29.912423    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:35.361663 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:35.361602    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:35.362181 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:35.361692    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:35.368849 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:35.368814    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:35.552102 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:35.552073    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fd8999687-gn9xg"
Jul 10 03:42:35.601366 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:35.601333    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56d469c765-zt56b"]
Jul 10 03:42:35.703034 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:35.702951    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:35.756936 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:35.756897    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:35.812965 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:35.812931    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:38.671613 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:38.671580    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:38.779970 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:38.779933    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:43.008531 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:43.008441    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:43.063105 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:43.063067    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:43.409767 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:43.409711    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:43.511278 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:43.511230    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:47.212856 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:47.212818    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:47.312953 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:47.312915    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:50.648918 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:50.648877    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:50.749691 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:50.749629    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:52.655722 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:52.655686    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq" event={"ID":"197c9c66-7a0e-4317-a816-b6db7422d5d6","Type":"ContainerStarted","Data":"7d23cdb51331f18a63a16d8b80650667f41928dc94d33bee53ce19b09ffffa8e"}
Jul 10 03:42:52.669457 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:52.669409    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wbcrq" podStartSLOduration=2.3154794450000002 podStartE2EDuration="25.669394689s" totalImagesPullingTime="23.353915244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:42:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:42:52.668822239 +0000 UTC m=+634.159818136" watchObservedRunningTime="2026-07-10 03:42:52.669394689 +0000 UTC m=+634.160390562"
Jul 10 03:42:57.603707 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:57.603644    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:42:57.704261 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:42:57.704224    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:00.628907 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:00.628871    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56d469c765-zt56b" podUID="0035e462-2069-481f-9900-d5f5b0fd4b9f" containerName="console" containerID="cri-o://0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa" gracePeriod=15
Jul 10 03:43:00.875796 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:00.875771    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:43:01.037572 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.037495    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-config\") pod \"0035e462-2069-481f-9900-d5f5b0fd4b9f\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") "
Jul 10 03:43:01.037572 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.037566    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-oauth-config\") pod \"0035e462-2069-481f-9900-d5f5b0fd4b9f\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") "
Jul 10 03:43:01.037825 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.037593    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-serving-cert\") pod \"0035e462-2069-481f-9900-d5f5b0fd4b9f\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") "
Jul 10 03:43:01.037825 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.037614    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-trusted-ca-bundle\") pod \"0035e462-2069-481f-9900-d5f5b0fd4b9f\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") "
Jul 10 03:43:01.037825 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.037629    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-oauth-serving-cert\") pod \"0035e462-2069-481f-9900-d5f5b0fd4b9f\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") "
Jul 10 03:43:01.037825 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.037683    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv9kp\" (UniqueName: \"kubernetes.io/projected/0035e462-2069-481f-9900-d5f5b0fd4b9f-kube-api-access-vv9kp\") pod \"0035e462-2069-481f-9900-d5f5b0fd4b9f\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") "
Jul 10 03:43:01.037825 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.037719    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-service-ca\") pod \"0035e462-2069-481f-9900-d5f5b0fd4b9f\" (UID: \"0035e462-2069-481f-9900-d5f5b0fd4b9f\") "
Jul 10 03:43:01.038090 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.038038    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-config" (OuterVolumeSpecName: "console-config") pod "0035e462-2069-481f-9900-d5f5b0fd4b9f" (UID: "0035e462-2069-481f-9900-d5f5b0fd4b9f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:43:01.038368 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.038342    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0035e462-2069-481f-9900-d5f5b0fd4b9f" (UID: "0035e462-2069-481f-9900-d5f5b0fd4b9f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:43:01.038368 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.038353    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0035e462-2069-481f-9900-d5f5b0fd4b9f" (UID: "0035e462-2069-481f-9900-d5f5b0fd4b9f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:43:01.038513 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.038392    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-service-ca" (OuterVolumeSpecName: "service-ca") pod "0035e462-2069-481f-9900-d5f5b0fd4b9f" (UID: "0035e462-2069-481f-9900-d5f5b0fd4b9f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:43:01.040024 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.039991    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0035e462-2069-481f-9900-d5f5b0fd4b9f" (UID: "0035e462-2069-481f-9900-d5f5b0fd4b9f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:43:01.040132 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.040054    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0035e462-2069-481f-9900-d5f5b0fd4b9f" (UID: "0035e462-2069-481f-9900-d5f5b0fd4b9f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:43:01.040132 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.040081    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0035e462-2069-481f-9900-d5f5b0fd4b9f-kube-api-access-vv9kp" (OuterVolumeSpecName: "kube-api-access-vv9kp") pod "0035e462-2069-481f-9900-d5f5b0fd4b9f" (UID: "0035e462-2069-481f-9900-d5f5b0fd4b9f"). InnerVolumeSpecName "kube-api-access-vv9kp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:43:01.138734 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.138695    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv9kp\" (UniqueName: \"kubernetes.io/projected/0035e462-2069-481f-9900-d5f5b0fd4b9f-kube-api-access-vv9kp\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:01.138734 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.138728    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:01.138734 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.138738    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-config\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:01.138734 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.138747    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-oauth-config\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:01.138999 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.138755    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0035e462-2069-481f-9900-d5f5b0fd4b9f-console-serving-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:01.138999 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.138764    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-trusted-ca-bundle\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:01.138999 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.138772    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0035e462-2069-481f-9900-d5f5b0fd4b9f-oauth-serving-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:01.704448 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.704416    2561 generic.go:356] "Generic (PLEG): container finished" podID="0035e462-2069-481f-9900-d5f5b0fd4b9f" containerID="0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa" exitCode=0
Jul 10 03:43:01.704857 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.704461    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d469c765-zt56b"
Jul 10 03:43:01.704857 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.704496    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d469c765-zt56b" event={"ID":"0035e462-2069-481f-9900-d5f5b0fd4b9f","Type":"ContainerDied","Data":"0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa"}
Jul 10 03:43:01.704857 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.704542    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d469c765-zt56b" event={"ID":"0035e462-2069-481f-9900-d5f5b0fd4b9f","Type":"ContainerDied","Data":"39458bcdff6fd65f446cdbaa3ccb375d28d1c650af5d68b0bb33cfe7756f9e83"}
Jul 10 03:43:01.704857 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.704561    2561 scope.go:117] "RemoveContainer" containerID="0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa"
Jul 10 03:43:01.715691 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.715670    2561 scope.go:117] "RemoveContainer" containerID="0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa"
Jul 10 03:43:01.716074 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:43:01.716052    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa\": container with ID starting with 0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa not found: ID does not exist" containerID="0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa"
Jul 10 03:43:01.716146 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.716084    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa"} err="failed to get container status \"0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa\": rpc error: code = NotFound desc = could not find container \"0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa\": container with ID starting with 0c18977b6f7ba9f06ccdd0a7acd21c507cb4f18b536fe0e0d7f43a579c296cfa not found: ID does not exist"
Jul 10 03:43:01.726396 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.726372    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56d469c765-zt56b"]
Jul 10 03:43:01.729957 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:01.729935    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56d469c765-zt56b"]
Jul 10 03:43:02.402969 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:02.402939    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:02.502498 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:02.502463    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:02.994592 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:02.994560    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0035e462-2069-481f-9900-d5f5b0fd4b9f" path="/var/lib/kubelet/pods/0035e462-2069-481f-9900-d5f5b0fd4b9f/volumes"
Jul 10 03:43:06.799345 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:06.799308    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:07.079812 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:07.079778    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:08.522909 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:08.522872    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:08.697530 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:08.697494    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:08.793364 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:08.793280    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:08.892574 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:08.892543    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:09.348603 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.348568    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-k29tl"]
Jul 10 03:43:09.348938 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.348924    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0035e462-2069-481f-9900-d5f5b0fd4b9f" containerName="console"
Jul 10 03:43:09.348990 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.348938    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0035e462-2069-481f-9900-d5f5b0fd4b9f" containerName="console"
Jul 10 03:43:09.349024 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.348993    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0035e462-2069-481f-9900-d5f5b0fd4b9f" containerName="console"
Jul 10 03:43:09.353108 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.353090    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k29tl"
Jul 10 03:43:09.355225 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.355205    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xpkv2\""
Jul 10 03:43:09.357990 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.357970    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-k29tl"]
Jul 10 03:43:09.382824 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.382797    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:09.402053 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.402028    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlbqn\" (UniqueName: \"kubernetes.io/projected/4f8181f0-f2d8-4831-9993-fc15642d3583-kube-api-access-jlbqn\") pod \"authorino-58cd955675-k29tl\" (UID: \"4f8181f0-f2d8-4831-9993-fc15642d3583\") " pod="kuadrant-system/authorino-58cd955675-k29tl"
Jul 10 03:43:09.480357 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.480330    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:09.502476 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.502455    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlbqn\" (UniqueName: \"kubernetes.io/projected/4f8181f0-f2d8-4831-9993-fc15642d3583-kube-api-access-jlbqn\") pod \"authorino-58cd955675-k29tl\" (UID: \"4f8181f0-f2d8-4831-9993-fc15642d3583\") " pod="kuadrant-system/authorino-58cd955675-k29tl"
Jul 10 03:43:09.513021 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.513002    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlbqn\" (UniqueName: \"kubernetes.io/projected/4f8181f0-f2d8-4831-9993-fc15642d3583-kube-api-access-jlbqn\") pod \"authorino-58cd955675-k29tl\" (UID: \"4f8181f0-f2d8-4831-9993-fc15642d3583\") " pod="kuadrant-system/authorino-58cd955675-k29tl"
Jul 10 03:43:09.518670 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.518632    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-j5v7m"]
Jul 10 03:43:09.522574 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.522557    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-j5v7m"
Jul 10 03:43:09.527293 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.527241    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-j5v7m"]
Jul 10 03:43:09.603494 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.603420    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vlm4\" (UniqueName: \"kubernetes.io/projected/8b928288-8ca6-441b-846e-7e0c5163200f-kube-api-access-6vlm4\") pod \"authorino-75d556bd77-j5v7m\" (UID: \"8b928288-8ca6-441b-846e-7e0c5163200f\") " pod="kuadrant-system/authorino-75d556bd77-j5v7m"
Jul 10 03:43:09.663036 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.663004    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k29tl"
Jul 10 03:43:09.704884 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.704850    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vlm4\" (UniqueName: \"kubernetes.io/projected/8b928288-8ca6-441b-846e-7e0c5163200f-kube-api-access-6vlm4\") pod \"authorino-75d556bd77-j5v7m\" (UID: \"8b928288-8ca6-441b-846e-7e0c5163200f\") " pod="kuadrant-system/authorino-75d556bd77-j5v7m"
Jul 10 03:43:09.714975 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.714952    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vlm4\" (UniqueName: \"kubernetes.io/projected/8b928288-8ca6-441b-846e-7e0c5163200f-kube-api-access-6vlm4\") pod \"authorino-75d556bd77-j5v7m\" (UID: \"8b928288-8ca6-441b-846e-7e0c5163200f\") " pod="kuadrant-system/authorino-75d556bd77-j5v7m"
Jul 10 03:43:09.792310 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.792284    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-k29tl"]
Jul 10 03:43:09.794680 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:43:09.794635    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f8181f0_f2d8_4831_9993_fc15642d3583.slice/crio-479ac5ed9c24d1bb3f846c60b9a6a0201cddd9cb94fb004ee5d55c41f5730796 WatchSource:0}: Error finding container 479ac5ed9c24d1bb3f846c60b9a6a0201cddd9cb94fb004ee5d55c41f5730796: Status 404 returned error can't find the container with id 479ac5ed9c24d1bb3f846c60b9a6a0201cddd9cb94fb004ee5d55c41f5730796
Jul 10 03:43:09.832593 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.832561    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-j5v7m"
Jul 10 03:43:09.963858 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:09.963830    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-j5v7m"]
Jul 10 03:43:09.965272 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:43:09.965243    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b928288_8ca6_441b_846e_7e0c5163200f.slice/crio-55b66ff45cf1f14a172a7981e440cebd0679155fb83e0f6701b9b88e9551901e WatchSource:0}: Error finding container 55b66ff45cf1f14a172a7981e440cebd0679155fb83e0f6701b9b88e9551901e: Status 404 returned error can't find the container with id 55b66ff45cf1f14a172a7981e440cebd0679155fb83e0f6701b9b88e9551901e
Jul 10 03:43:10.758900 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:10.758808    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k29tl" event={"ID":"4f8181f0-f2d8-4831-9993-fc15642d3583","Type":"ContainerStarted","Data":"479ac5ed9c24d1bb3f846c60b9a6a0201cddd9cb94fb004ee5d55c41f5730796"}
Jul 10 03:43:10.762956 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:10.762884    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-j5v7m" event={"ID":"8b928288-8ca6-441b-846e-7e0c5163200f","Type":"ContainerStarted","Data":"55b66ff45cf1f14a172a7981e440cebd0679155fb83e0f6701b9b88e9551901e"}
Jul 10 03:43:12.612007 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:12.611953    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:12.667030 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:12.666977    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:13.780409 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:13.780317    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k29tl" event={"ID":"4f8181f0-f2d8-4831-9993-fc15642d3583","Type":"ContainerStarted","Data":"9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac"}
Jul 10 03:43:13.781860 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:13.781834    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-j5v7m" event={"ID":"8b928288-8ca6-441b-846e-7e0c5163200f","Type":"ContainerStarted","Data":"1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6"}
Jul 10 03:43:13.794939 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:13.794891    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-k29tl" podStartSLOduration=1.240302091 podStartE2EDuration="4.794877974s" totalImagesPullingTime="3.554575883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:43:13.793013482 +0000 UTC m=+655.284009370" watchObservedRunningTime="2026-07-10 03:43:13.794877974 +0000 UTC m=+655.285873847"
Jul 10 03:43:13.806788 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:13.806731    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-j5v7m" podStartSLOduration=1.411201026 podStartE2EDuration="4.806712518s" totalImagesPullingTime="3.395511492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:43:13.805144775 +0000 UTC m=+655.296140659" watchObservedRunningTime="2026-07-10 03:43:13.806712518 +0000 UTC m=+655.297708391"
Jul 10 03:43:13.826013 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:13.825982    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-k29tl"]
Jul 10 03:43:15.791998 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:15.791955    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-k29tl" podUID="4f8181f0-f2d8-4831-9993-fc15642d3583" containerName="authorino" containerID="cri-o://9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac" gracePeriod=30
Jul 10 03:43:16.038530 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.038504    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k29tl"
Jul 10 03:43:16.064757 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.064689    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlbqn\" (UniqueName: \"kubernetes.io/projected/4f8181f0-f2d8-4831-9993-fc15642d3583-kube-api-access-jlbqn\") pod \"4f8181f0-f2d8-4831-9993-fc15642d3583\" (UID: \"4f8181f0-f2d8-4831-9993-fc15642d3583\") "
Jul 10 03:43:16.074430 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.074391    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f8181f0-f2d8-4831-9993-fc15642d3583-kube-api-access-jlbqn" (OuterVolumeSpecName: "kube-api-access-jlbqn") pod "4f8181f0-f2d8-4831-9993-fc15642d3583" (UID: "4f8181f0-f2d8-4831-9993-fc15642d3583"). InnerVolumeSpecName "kube-api-access-jlbqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:43:16.165894 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.165858    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlbqn\" (UniqueName: \"kubernetes.io/projected/4f8181f0-f2d8-4831-9993-fc15642d3583-kube-api-access-jlbqn\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:16.798491 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.798458    2561 generic.go:356] "Generic (PLEG): container finished" podID="4f8181f0-f2d8-4831-9993-fc15642d3583" containerID="9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac" exitCode=0
Jul 10 03:43:16.798948 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.798526    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k29tl"
Jul 10 03:43:16.798948 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.798537    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k29tl" event={"ID":"4f8181f0-f2d8-4831-9993-fc15642d3583","Type":"ContainerDied","Data":"9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac"}
Jul 10 03:43:16.798948 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.798585    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k29tl" event={"ID":"4f8181f0-f2d8-4831-9993-fc15642d3583","Type":"ContainerDied","Data":"479ac5ed9c24d1bb3f846c60b9a6a0201cddd9cb94fb004ee5d55c41f5730796"}
Jul 10 03:43:16.798948 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.798603    2561 scope.go:117] "RemoveContainer" containerID="9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac"
Jul 10 03:43:16.807378 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.807362    2561 scope.go:117] "RemoveContainer" containerID="9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac"
Jul 10 03:43:16.807692 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:43:16.807669    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac\": container with ID starting with 9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac not found: ID does not exist" containerID="9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac"
Jul 10 03:43:16.807806 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.807697    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac"} err="failed to get container status \"9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac\": rpc error: code = NotFound desc = could not find container \"9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac\": container with ID starting with 9a3617d16524f6a2af57b7be535985581b1e03c0a7e61cabd8f085acf1b617ac not found: ID does not exist"
Jul 10 03:43:16.826032 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.825997    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-k29tl"]
Jul 10 03:43:16.827970 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.827945    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-k29tl"]
Jul 10 03:43:16.859333 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.859301    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:16.994072 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:16.994042    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f8181f0-f2d8-4831-9993-fc15642d3583" path="/var/lib/kubelet/pods/4f8181f0-f2d8-4831-9993-fc15642d3583/volumes"
Jul 10 03:43:20.811432 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:20.811397    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:20.912088 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:20.912054    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:24.663696 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:24.663668    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:24.768878 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:24.768839    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:25.250985 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:25.250944    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:25.402740 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:25.402696    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:25.504925 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:25.504819    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:26.006198 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:26.006155    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:26.106308 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:26.106276    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:36.955832 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:36.955774    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:37.052592 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.052548    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:37.184608 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.184570    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-59qd8"]
Jul 10 03:43:37.185010 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.184993    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f8181f0-f2d8-4831-9993-fc15642d3583" containerName="authorino"
Jul 10 03:43:37.185096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.185012    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f8181f0-f2d8-4831-9993-fc15642d3583" containerName="authorino"
Jul 10 03:43:37.185150 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.185113    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f8181f0-f2d8-4831-9993-fc15642d3583" containerName="authorino"
Jul 10 03:43:37.233770 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.233600    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-59qd8"]
Jul 10 03:43:37.233770 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.233736    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-59qd8"
Jul 10 03:43:37.360910 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.360873    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc4zz\" (UniqueName: \"kubernetes.io/projected/ab3788cc-e339-4878-99e9-44f3aae74d86-kube-api-access-hc4zz\") pod \"authorino-5d4b67c446-59qd8\" (UID: \"ab3788cc-e339-4878-99e9-44f3aae74d86\") " pod="kuadrant-system/authorino-5d4b67c446-59qd8"
Jul 10 03:43:37.361083 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.360967    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-59qd8"]
Jul 10 03:43:37.361194 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:43:37.361175    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-hc4zz], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-59qd8" podUID="ab3788cc-e339-4878-99e9-44f3aae74d86"
Jul 10 03:43:37.385833 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.385797    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-jhf6d"]
Jul 10 03:43:37.389298 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.389282    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-jhf6d"
Jul 10 03:43:37.391550 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.391528    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 10 03:43:37.398247 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.398227    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-jhf6d"]
Jul 10 03:43:37.462480 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.462441    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a834477a-df94-4faf-b106-fce9b1945e55-tls-cert\") pod \"authorino-76877994-jhf6d\" (UID: \"a834477a-df94-4faf-b106-fce9b1945e55\") " pod="kuadrant-system/authorino-76877994-jhf6d"
Jul 10 03:43:37.462687 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.462485    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wksd\" (UniqueName: \"kubernetes.io/projected/a834477a-df94-4faf-b106-fce9b1945e55-kube-api-access-7wksd\") pod \"authorino-76877994-jhf6d\" (UID: \"a834477a-df94-4faf-b106-fce9b1945e55\") " pod="kuadrant-system/authorino-76877994-jhf6d"
Jul 10 03:43:37.462687 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.462513    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc4zz\" (UniqueName: \"kubernetes.io/projected/ab3788cc-e339-4878-99e9-44f3aae74d86-kube-api-access-hc4zz\") pod \"authorino-5d4b67c446-59qd8\" (UID: \"ab3788cc-e339-4878-99e9-44f3aae74d86\") " pod="kuadrant-system/authorino-5d4b67c446-59qd8"
Jul 10 03:43:37.475560 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.475531    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc4zz\" (UniqueName: \"kubernetes.io/projected/ab3788cc-e339-4878-99e9-44f3aae74d86-kube-api-access-hc4zz\") pod \"authorino-5d4b67c446-59qd8\" (UID: \"ab3788cc-e339-4878-99e9-44f3aae74d86\") " pod="kuadrant-system/authorino-5d4b67c446-59qd8"
Jul 10 03:43:37.504441 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.504355    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-jhf6d"]
Jul 10 03:43:37.504645 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:43:37.504623    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7wksd tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-jhf6d" podUID="a834477a-df94-4faf-b106-fce9b1945e55"
Jul 10 03:43:37.559819 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.559782    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-86f5f5c6c-7pmwb"]
Jul 10 03:43:37.563823 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.563800    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86f5f5c6c-7pmwb"
Jul 10 03:43:37.563947 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.563882    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wksd\" (UniqueName: \"kubernetes.io/projected/a834477a-df94-4faf-b106-fce9b1945e55-kube-api-access-7wksd\") pod \"authorino-76877994-jhf6d\" (UID: \"a834477a-df94-4faf-b106-fce9b1945e55\") " pod="kuadrant-system/authorino-76877994-jhf6d"
Jul 10 03:43:37.564001 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.563968    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a834477a-df94-4faf-b106-fce9b1945e55-tls-cert\") pod \"authorino-76877994-jhf6d\" (UID: \"a834477a-df94-4faf-b106-fce9b1945e55\") " pod="kuadrant-system/authorino-76877994-jhf6d"
Jul 10 03:43:37.566281 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.566257    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a834477a-df94-4faf-b106-fce9b1945e55-tls-cert\") pod \"authorino-76877994-jhf6d\" (UID: \"a834477a-df94-4faf-b106-fce9b1945e55\") " pod="kuadrant-system/authorino-76877994-jhf6d"
Jul 10 03:43:37.584907 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.584872    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86f5f5c6c-7pmwb"]
Jul 10 03:43:37.588608 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.588588    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wksd\" (UniqueName: \"kubernetes.io/projected/a834477a-df94-4faf-b106-fce9b1945e55-kube-api-access-7wksd\") pod \"authorino-76877994-jhf6d\" (UID: \"a834477a-df94-4faf-b106-fce9b1945e55\") " pod="kuadrant-system/authorino-76877994-jhf6d"
Jul 10 03:43:37.664380 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.664346    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxjpd\" (UniqueName: \"kubernetes.io/projected/2f24b82c-e9f8-4ab9-bc03-47c5849bda12-kube-api-access-sxjpd\") pod \"authorino-86f5f5c6c-7pmwb\" (UID: \"2f24b82c-e9f8-4ab9-bc03-47c5849bda12\") " pod="kuadrant-system/authorino-86f5f5c6c-7pmwb"
Jul 10 03:43:37.664540 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.664386    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2f24b82c-e9f8-4ab9-bc03-47c5849bda12-tls-cert\") pod \"authorino-86f5f5c6c-7pmwb\" (UID: \"2f24b82c-e9f8-4ab9-bc03-47c5849bda12\") " pod="kuadrant-system/authorino-86f5f5c6c-7pmwb"
Jul 10 03:43:37.765255 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.765156    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxjpd\" (UniqueName: \"kubernetes.io/projected/2f24b82c-e9f8-4ab9-bc03-47c5849bda12-kube-api-access-sxjpd\") pod \"authorino-86f5f5c6c-7pmwb\" (UID: \"2f24b82c-e9f8-4ab9-bc03-47c5849bda12\") " pod="kuadrant-system/authorino-86f5f5c6c-7pmwb"
Jul 10 03:43:37.765255 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.765215    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2f24b82c-e9f8-4ab9-bc03-47c5849bda12-tls-cert\") pod \"authorino-86f5f5c6c-7pmwb\" (UID: \"2f24b82c-e9f8-4ab9-bc03-47c5849bda12\") " pod="kuadrant-system/authorino-86f5f5c6c-7pmwb"
Jul 10 03:43:37.767641 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.767608    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2f24b82c-e9f8-4ab9-bc03-47c5849bda12-tls-cert\") pod \"authorino-86f5f5c6c-7pmwb\" (UID: \"2f24b82c-e9f8-4ab9-bc03-47c5849bda12\") " pod="kuadrant-system/authorino-86f5f5c6c-7pmwb"
Jul 10 03:43:37.772710 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.772687    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxjpd\" (UniqueName: \"kubernetes.io/projected/2f24b82c-e9f8-4ab9-bc03-47c5849bda12-kube-api-access-sxjpd\") pod \"authorino-86f5f5c6c-7pmwb\" (UID: \"2f24b82c-e9f8-4ab9-bc03-47c5849bda12\") " pod="kuadrant-system/authorino-86f5f5c6c-7pmwb"
Jul 10 03:43:37.881613 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.881577    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86f5f5c6c-7pmwb"
Jul 10 03:43:37.906903 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.906873    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-jhf6d"
Jul 10 03:43:37.907064 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.906873    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-59qd8"
Jul 10 03:43:37.914485 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.914451    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-jhf6d"
Jul 10 03:43:37.917246 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:37.917225    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-59qd8"
Jul 10 03:43:38.015446 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.015362    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86f5f5c6c-7pmwb"]
Jul 10 03:43:38.018356 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:43:38.018332    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f24b82c_e9f8_4ab9_bc03_47c5849bda12.slice/crio-9cc88143e5544b77852814158253a1e772b255fa5060f7eb813eb69742bbcf8b WatchSource:0}: Error finding container 9cc88143e5544b77852814158253a1e772b255fa5060f7eb813eb69742bbcf8b: Status 404 returned error can't find the container with id 9cc88143e5544b77852814158253a1e772b255fa5060f7eb813eb69742bbcf8b
Jul 10 03:43:38.019782 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.019764    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 03:43:38.069228 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.069197    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a834477a-df94-4faf-b106-fce9b1945e55-tls-cert\") pod \"a834477a-df94-4faf-b106-fce9b1945e55\" (UID: \"a834477a-df94-4faf-b106-fce9b1945e55\") "
Jul 10 03:43:38.069448 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.069245    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc4zz\" (UniqueName: \"kubernetes.io/projected/ab3788cc-e339-4878-99e9-44f3aae74d86-kube-api-access-hc4zz\") pod \"ab3788cc-e339-4878-99e9-44f3aae74d86\" (UID: \"ab3788cc-e339-4878-99e9-44f3aae74d86\") "
Jul 10 03:43:38.069448 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.069296    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wksd\" (UniqueName: \"kubernetes.io/projected/a834477a-df94-4faf-b106-fce9b1945e55-kube-api-access-7wksd\") pod \"a834477a-df94-4faf-b106-fce9b1945e55\" (UID: \"a834477a-df94-4faf-b106-fce9b1945e55\") "
Jul 10 03:43:38.071503 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.071473    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a834477a-df94-4faf-b106-fce9b1945e55-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a834477a-df94-4faf-b106-fce9b1945e55" (UID: "a834477a-df94-4faf-b106-fce9b1945e55"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:43:38.071503 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.071480    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a834477a-df94-4faf-b106-fce9b1945e55-kube-api-access-7wksd" (OuterVolumeSpecName: "kube-api-access-7wksd") pod "a834477a-df94-4faf-b106-fce9b1945e55" (UID: "a834477a-df94-4faf-b106-fce9b1945e55"). InnerVolumeSpecName "kube-api-access-7wksd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:43:38.071644 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.071515    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab3788cc-e339-4878-99e9-44f3aae74d86-kube-api-access-hc4zz" (OuterVolumeSpecName: "kube-api-access-hc4zz") pod "ab3788cc-e339-4878-99e9-44f3aae74d86" (UID: "ab3788cc-e339-4878-99e9-44f3aae74d86"). InnerVolumeSpecName "kube-api-access-hc4zz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:43:38.170533 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.170497    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc4zz\" (UniqueName: \"kubernetes.io/projected/ab3788cc-e339-4878-99e9-44f3aae74d86-kube-api-access-hc4zz\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:38.170533 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.170529    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wksd\" (UniqueName: \"kubernetes.io/projected/a834477a-df94-4faf-b106-fce9b1945e55-kube-api-access-7wksd\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:38.170533 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.170541    2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a834477a-df94-4faf-b106-fce9b1945e55-tls-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:38.912958 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.912923    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86f5f5c6c-7pmwb" event={"ID":"2f24b82c-e9f8-4ab9-bc03-47c5849bda12","Type":"ContainerStarted","Data":"e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c"}
Jul 10 03:43:38.913158 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.912959    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-59qd8"
Jul 10 03:43:38.913158 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.912971    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86f5f5c6c-7pmwb" event={"ID":"2f24b82c-e9f8-4ab9-bc03-47c5849bda12","Type":"ContainerStarted","Data":"9cc88143e5544b77852814158253a1e772b255fa5060f7eb813eb69742bbcf8b"}
Jul 10 03:43:38.913274 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.913215    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-jhf6d"
Jul 10 03:43:38.927451 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.927404    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-86f5f5c6c-7pmwb" podStartSLOduration=1.6129375700000002 podStartE2EDuration="1.927389757s" totalImagesPullingTime="314.452187ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:43:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:43:38.926722946 +0000 UTC m=+680.417718819" watchObservedRunningTime="2026-07-10 03:43:38.927389757 +0000 UTC m=+680.418385632"
Jul 10 03:43:38.957418 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.957385    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-59qd8"]
Jul 10 03:43:38.971934 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.971890    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-j5v7m"]
Jul 10 03:43:38.973296 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.973095    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-j5v7m" podUID="8b928288-8ca6-441b-846e-7e0c5163200f" containerName="authorino" containerID="cri-o://1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6" gracePeriod=30
Jul 10 03:43:38.976377 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.976331    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-59qd8"]
Jul 10 03:43:38.996197 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.996168    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab3788cc-e339-4878-99e9-44f3aae74d86" path="/var/lib/kubelet/pods/ab3788cc-e339-4878-99e9-44f3aae74d86/volumes"
Jul 10 03:43:38.996373 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:38.996357    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-jhf6d"]
Jul 10 03:43:39.000374 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.000349    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-jhf6d"]
Jul 10 03:43:39.223795 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.223771    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-j5v7m"
Jul 10 03:43:39.381555 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.381527    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vlm4\" (UniqueName: \"kubernetes.io/projected/8b928288-8ca6-441b-846e-7e0c5163200f-kube-api-access-6vlm4\") pod \"8b928288-8ca6-441b-846e-7e0c5163200f\" (UID: \"8b928288-8ca6-441b-846e-7e0c5163200f\") "
Jul 10 03:43:39.383619 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.383594    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b928288-8ca6-441b-846e-7e0c5163200f-kube-api-access-6vlm4" (OuterVolumeSpecName: "kube-api-access-6vlm4") pod "8b928288-8ca6-441b-846e-7e0c5163200f" (UID: "8b928288-8ca6-441b-846e-7e0c5163200f"). InnerVolumeSpecName "kube-api-access-6vlm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:43:39.482808 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.482723    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vlm4\" (UniqueName: \"kubernetes.io/projected/8b928288-8ca6-441b-846e-7e0c5163200f-kube-api-access-6vlm4\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:39.919715 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.919682    2561 generic.go:356] "Generic (PLEG): container finished" podID="8b928288-8ca6-441b-846e-7e0c5163200f" containerID="1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6" exitCode=0
Jul 10 03:43:39.919891 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.919727    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-j5v7m"
Jul 10 03:43:39.919891 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.919770    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-j5v7m" event={"ID":"8b928288-8ca6-441b-846e-7e0c5163200f","Type":"ContainerDied","Data":"1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6"}
Jul 10 03:43:39.919891 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.919812    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-j5v7m" event={"ID":"8b928288-8ca6-441b-846e-7e0c5163200f","Type":"ContainerDied","Data":"55b66ff45cf1f14a172a7981e440cebd0679155fb83e0f6701b9b88e9551901e"}
Jul 10 03:43:39.919891 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.919834    2561 scope.go:117] "RemoveContainer" containerID="1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6"
Jul 10 03:43:39.928510 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.928490    2561 scope.go:117] "RemoveContainer" containerID="1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6"
Jul 10 03:43:39.928774 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:43:39.928755    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6\": container with ID starting with 1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6 not found: ID does not exist" containerID="1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6"
Jul 10 03:43:39.928850 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.928782    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6"} err="failed to get container status \"1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6\": rpc error: code = NotFound desc = could not find container \"1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6\": container with ID starting with 1d68e6d2804de0eaf8649b73215f1c15a90d0682adef12a54e8dabcfb1c5a2c6 not found: ID does not exist"
Jul 10 03:43:39.940813 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.940768    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-j5v7m"]
Jul 10 03:43:39.943925 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:39.943905    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-j5v7m"]
Jul 10 03:43:40.993635 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:40.993597    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b928288-8ca6-441b-846e-7e0c5163200f" path="/var/lib/kubelet/pods/8b928288-8ca6-441b-846e-7e0c5163200f/volumes"
Jul 10 03:43:40.994024 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:40.993983    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a834477a-df94-4faf-b106-fce9b1945e55" path="/var/lib/kubelet/pods/a834477a-df94-4faf-b106-fce9b1945e55/volumes"
Jul 10 03:43:43.327280 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.327246    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:43.344927 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.344898    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c565d7fc-qdmdc"]
Jul 10 03:43:43.345263 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.345251    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b928288-8ca6-441b-846e-7e0c5163200f" containerName="authorino"
Jul 10 03:43:43.345306 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.345264    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b928288-8ca6-441b-846e-7e0c5163200f" containerName="authorino"
Jul 10 03:43:43.345338 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.345323    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b928288-8ca6-441b-846e-7e0c5163200f" containerName="authorino"
Jul 10 03:43:43.349924 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.349890    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c565d7fc-qdmdc"
Jul 10 03:43:43.353906 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.353877    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 10 03:43:43.354058 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.353892    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ngm4n\""
Jul 10 03:43:43.354132 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.354118    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 10 03:43:43.357396 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.357376    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c565d7fc-qdmdc"]
Jul 10 03:43:43.427445 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.427414    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:43.478303 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.478273    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:43.512980 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.512948    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f864f4b59-m79h4"]
Jul 10 03:43:43.516502 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.516484    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f864f4b59-m79h4"
Jul 10 03:43:43.517643 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.517624    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t8bb\" (UniqueName: \"kubernetes.io/projected/856f97fd-e348-4dab-9d17-a4cbe245f22a-kube-api-access-7t8bb\") pod \"maas-controller-c565d7fc-qdmdc\" (UID: \"856f97fd-e348-4dab-9d17-a4cbe245f22a\") " pod="opendatahub/maas-controller-c565d7fc-qdmdc"
Jul 10 03:43:43.517736 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.517712    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/856f97fd-e348-4dab-9d17-a4cbe245f22a-webhook-cert\") pod \"maas-controller-c565d7fc-qdmdc\" (UID: \"856f97fd-e348-4dab-9d17-a4cbe245f22a\") " pod="opendatahub/maas-controller-c565d7fc-qdmdc"
Jul 10 03:43:43.525573 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.525551    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f864f4b59-m79h4"]
Jul 10 03:43:43.618753 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.618641    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0863097f-adfe-46e0-a4df-462d9161934b-webhook-cert\") pod \"maas-controller-6f864f4b59-m79h4\" (UID: \"0863097f-adfe-46e0-a4df-462d9161934b\") " pod="opendatahub/maas-controller-6f864f4b59-m79h4"
Jul 10 03:43:43.618912 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.618752    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/856f97fd-e348-4dab-9d17-a4cbe245f22a-webhook-cert\") pod \"maas-controller-c565d7fc-qdmdc\" (UID: \"856f97fd-e348-4dab-9d17-a4cbe245f22a\") " pod="opendatahub/maas-controller-c565d7fc-qdmdc"
Jul 10 03:43:43.618912 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.618785    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw8gc\" (UniqueName: \"kubernetes.io/projected/0863097f-adfe-46e0-a4df-462d9161934b-kube-api-access-pw8gc\") pod \"maas-controller-6f864f4b59-m79h4\" (UID: \"0863097f-adfe-46e0-a4df-462d9161934b\") " pod="opendatahub/maas-controller-6f864f4b59-m79h4"
Jul 10 03:43:43.618912 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.618832    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t8bb\" (UniqueName: \"kubernetes.io/projected/856f97fd-e348-4dab-9d17-a4cbe245f22a-kube-api-access-7t8bb\") pod \"maas-controller-c565d7fc-qdmdc\" (UID: \"856f97fd-e348-4dab-9d17-a4cbe245f22a\") " pod="opendatahub/maas-controller-c565d7fc-qdmdc"
Jul 10 03:43:43.621219 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.621190    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/856f97fd-e348-4dab-9d17-a4cbe245f22a-webhook-cert\") pod \"maas-controller-c565d7fc-qdmdc\" (UID: \"856f97fd-e348-4dab-9d17-a4cbe245f22a\") " pod="opendatahub/maas-controller-c565d7fc-qdmdc"
Jul 10 03:43:43.625817 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.625792    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t8bb\" (UniqueName: \"kubernetes.io/projected/856f97fd-e348-4dab-9d17-a4cbe245f22a-kube-api-access-7t8bb\") pod \"maas-controller-c565d7fc-qdmdc\" (UID: \"856f97fd-e348-4dab-9d17-a4cbe245f22a\") " pod="opendatahub/maas-controller-c565d7fc-qdmdc"
Jul 10 03:43:43.663463 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.663434    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c565d7fc-qdmdc"
Jul 10 03:43:43.719674 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.719613    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0863097f-adfe-46e0-a4df-462d9161934b-webhook-cert\") pod \"maas-controller-6f864f4b59-m79h4\" (UID: \"0863097f-adfe-46e0-a4df-462d9161934b\") " pod="opendatahub/maas-controller-6f864f4b59-m79h4"
Jul 10 03:43:43.719831 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.719812    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw8gc\" (UniqueName: \"kubernetes.io/projected/0863097f-adfe-46e0-a4df-462d9161934b-kube-api-access-pw8gc\") pod \"maas-controller-6f864f4b59-m79h4\" (UID: \"0863097f-adfe-46e0-a4df-462d9161934b\") " pod="opendatahub/maas-controller-6f864f4b59-m79h4"
Jul 10 03:43:43.722220 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.722158    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0863097f-adfe-46e0-a4df-462d9161934b-webhook-cert\") pod \"maas-controller-6f864f4b59-m79h4\" (UID: \"0863097f-adfe-46e0-a4df-462d9161934b\") " pod="opendatahub/maas-controller-6f864f4b59-m79h4"
Jul 10 03:43:43.728603 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.728571    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw8gc\" (UniqueName: \"kubernetes.io/projected/0863097f-adfe-46e0-a4df-462d9161934b-kube-api-access-pw8gc\") pod \"maas-controller-6f864f4b59-m79h4\" (UID: \"0863097f-adfe-46e0-a4df-462d9161934b\") " pod="opendatahub/maas-controller-6f864f4b59-m79h4"
Jul 10 03:43:43.797091 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.797063    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c565d7fc-qdmdc"]
Jul 10 03:43:43.798562 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:43:43.798527    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod856f97fd_e348_4dab_9d17_a4cbe245f22a.slice/crio-2364de19c7ba568be08dcf8ca3a48eabb7a121b1f94ac0f9fa9eb78500ad6876 WatchSource:0}: Error finding container 2364de19c7ba568be08dcf8ca3a48eabb7a121b1f94ac0f9fa9eb78500ad6876: Status 404 returned error can't find the container with id 2364de19c7ba568be08dcf8ca3a48eabb7a121b1f94ac0f9fa9eb78500ad6876
Jul 10 03:43:43.827821 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.827800    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f864f4b59-m79h4"
Jul 10 03:43:43.941895 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.941852    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c565d7fc-qdmdc" event={"ID":"856f97fd-e348-4dab-9d17-a4cbe245f22a","Type":"ContainerStarted","Data":"2364de19c7ba568be08dcf8ca3a48eabb7a121b1f94ac0f9fa9eb78500ad6876"}
Jul 10 03:43:43.981978 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:43.981952    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f864f4b59-m79h4"]
Jul 10 03:43:43.983708 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:43:43.983678    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0863097f_adfe_46e0_a4df_462d9161934b.slice/crio-d85a4fa60984e1867c2858500f23e5ef8e07e859ac3829f3d813119838dd4d5f WatchSource:0}: Error finding container d85a4fa60984e1867c2858500f23e5ef8e07e859ac3829f3d813119838dd4d5f: Status 404 returned error can't find the container with id d85a4fa60984e1867c2858500f23e5ef8e07e859ac3829f3d813119838dd4d5f
Jul 10 03:43:44.954893 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:44.954764    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f864f4b59-m79h4" event={"ID":"0863097f-adfe-46e0-a4df-462d9161934b","Type":"ContainerStarted","Data":"d85a4fa60984e1867c2858500f23e5ef8e07e859ac3829f3d813119838dd4d5f"}
Jul 10 03:43:46.361099 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:46.361069    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:46.461041 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:46.460962    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:43:46.967089 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:46.967050    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c565d7fc-qdmdc" event={"ID":"856f97fd-e348-4dab-9d17-a4cbe245f22a","Type":"ContainerStarted","Data":"4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9"}
Jul 10 03:43:46.967285 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:46.967157    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c565d7fc-qdmdc"
Jul 10 03:43:46.968625 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:46.968599    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f864f4b59-m79h4" event={"ID":"0863097f-adfe-46e0-a4df-462d9161934b","Type":"ContainerStarted","Data":"e54401e873bb6b01b40f1aaf9ab9f8fcb67abe2790f8593639f647f1ca27d8f4"}
Jul 10 03:43:46.968758 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:46.968674    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f864f4b59-m79h4"
Jul 10 03:43:46.986641 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:46.986594    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c565d7fc-qdmdc" podStartSLOduration=1.5921260849999999 podStartE2EDuration="3.986582223s" totalImagesPullingTime="2.394456138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:43:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:43:46.984798761 +0000 UTC m=+688.475794661" watchObservedRunningTime="2026-07-10 03:43:46.986582223 +0000 UTC m=+688.477578095"
Jul 10 03:43:47.002471 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:47.002426    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f864f4b59-m79h4" podStartSLOduration=1.829348996 podStartE2EDuration="4.002413735s" totalImagesPullingTime="2.173064739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:43:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:43:47.001035257 +0000 UTC m=+688.492031130" watchObservedRunningTime="2026-07-10 03:43:47.002413735 +0000 UTC m=+688.493409608"
Jul 10 03:43:57.977088 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:57.977053    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c565d7fc-qdmdc"
Jul 10 03:43:57.977470 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:57.977256    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f864f4b59-m79h4"
Jul 10 03:43:58.030351 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:58.030307    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c565d7fc-qdmdc"]
Jul 10 03:43:58.030569 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:58.030540    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-c565d7fc-qdmdc" podUID="856f97fd-e348-4dab-9d17-a4cbe245f22a" containerName="manager" containerID="cri-o://4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9" gracePeriod=30
Jul 10 03:43:58.276272 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:58.276241    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c565d7fc-qdmdc"
Jul 10 03:43:58.338513 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:58.338479    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/856f97fd-e348-4dab-9d17-a4cbe245f22a-webhook-cert\") pod \"856f97fd-e348-4dab-9d17-a4cbe245f22a\" (UID: \"856f97fd-e348-4dab-9d17-a4cbe245f22a\") "
Jul 10 03:43:58.338676 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:58.338543    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t8bb\" (UniqueName: \"kubernetes.io/projected/856f97fd-e348-4dab-9d17-a4cbe245f22a-kube-api-access-7t8bb\") pod \"856f97fd-e348-4dab-9d17-a4cbe245f22a\" (UID: \"856f97fd-e348-4dab-9d17-a4cbe245f22a\") "
Jul 10 03:43:58.340730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:58.340697    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/856f97fd-e348-4dab-9d17-a4cbe245f22a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "856f97fd-e348-4dab-9d17-a4cbe245f22a" (UID: "856f97fd-e348-4dab-9d17-a4cbe245f22a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:43:58.340839 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:58.340793    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/856f97fd-e348-4dab-9d17-a4cbe245f22a-kube-api-access-7t8bb" (OuterVolumeSpecName: "kube-api-access-7t8bb") pod "856f97fd-e348-4dab-9d17-a4cbe245f22a" (UID: "856f97fd-e348-4dab-9d17-a4cbe245f22a"). InnerVolumeSpecName "kube-api-access-7t8bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:43:58.439379 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:58.439346    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/856f97fd-e348-4dab-9d17-a4cbe245f22a-webhook-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:58.439379 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:58.439374    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t8bb\" (UniqueName: \"kubernetes.io/projected/856f97fd-e348-4dab-9d17-a4cbe245f22a-kube-api-access-7t8bb\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:43:59.035517 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:59.035480    2561 generic.go:356] "Generic (PLEG): container finished" podID="856f97fd-e348-4dab-9d17-a4cbe245f22a" containerID="4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9" exitCode=0
Jul 10 03:43:59.035920 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:59.035530    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c565d7fc-qdmdc"
Jul 10 03:43:59.035920 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:59.035567    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c565d7fc-qdmdc" event={"ID":"856f97fd-e348-4dab-9d17-a4cbe245f22a","Type":"ContainerDied","Data":"4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9"}
Jul 10 03:43:59.035920 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:59.035618    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c565d7fc-qdmdc" event={"ID":"856f97fd-e348-4dab-9d17-a4cbe245f22a","Type":"ContainerDied","Data":"2364de19c7ba568be08dcf8ca3a48eabb7a121b1f94ac0f9fa9eb78500ad6876"}
Jul 10 03:43:59.035920 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:59.035639    2561 scope.go:117] "RemoveContainer" containerID="4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9"
Jul 10 03:43:59.044586 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:59.044569    2561 scope.go:117] "RemoveContainer" containerID="4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9"
Jul 10 03:43:59.044848 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:43:59.044827    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9\": container with ID starting with 4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9 not found: ID does not exist" containerID="4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9"
Jul 10 03:43:59.044902 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:59.044859    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9"} err="failed to get container status \"4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9\": rpc error: code = NotFound desc = could not find container \"4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9\": container with ID starting with 4c68e753f4317349f372ff8ec49e54996df3768abee1438ddc76ef71105092b9 not found: ID does not exist"
Jul 10 03:43:59.050957 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:59.050937    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c565d7fc-qdmdc"]
Jul 10 03:43:59.056395 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:43:59.056374    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c565d7fc-qdmdc"]
Jul 10 03:44:00.993933 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:00.993903    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="856f97fd-e348-4dab-9d17-a4cbe245f22a" path="/var/lib/kubelet/pods/856f97fd-e348-4dab-9d17-a4cbe245f22a/volumes"
Jul 10 03:44:17.779177 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:17.779084    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:44:17.877957 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:17.877922    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:44:18.298441 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:18.298404    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:44:18.439096 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:18.439063    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:44:18.942004 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:18.941184    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:44:19.035876 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.035840    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:44:19.051190 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.051159    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-rnqxm"]
Jul 10 03:44:19.051624 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.051598    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="856f97fd-e348-4dab-9d17-a4cbe245f22a" containerName="manager"
Jul 10 03:44:19.051624 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.051624    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="856f97fd-e348-4dab-9d17-a4cbe245f22a" containerName="manager"
Jul 10 03:44:19.051827 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.051767    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="856f97fd-e348-4dab-9d17-a4cbe245f22a" containerName="manager"
Jul 10 03:44:19.057863 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.057841    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm"
Jul 10 03:44:19.060398 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.060377    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 10 03:44:19.065713 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.065691    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-rnqxm"]
Jul 10 03:44:19.071525 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.071503    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-ldtg2"]
Jul 10 03:44:19.076022 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.076005    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-ldtg2"
Jul 10 03:44:19.078002 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.077983    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-m5wjd\""
Jul 10 03:44:19.085232 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.085212    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-ldtg2"]
Jul 10 03:44:19.116037 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.116004    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98q4s\" (UniqueName: \"kubernetes.io/projected/fb1e2f13-1834-43a2-8f83-786caa86eaa2-kube-api-access-98q4s\") pod \"payload-processing-58f86c7798-ldtg2\" (UID: \"fb1e2f13-1834-43a2-8f83-786caa86eaa2\") " pod="openshift-ingress/payload-processing-58f86c7798-ldtg2"
Jul 10 03:44:19.116037 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.116040    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11c1301b-ab4e-490b-8012-e973bd2b00be-plugins-config-volume\") pod \"payload-pre-processing-7964457696-rnqxm\" (UID: \"11c1301b-ab4e-490b-8012-e973bd2b00be\") " pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm"
Jul 10 03:44:19.116263 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.116118    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r94k7\" (UniqueName: \"kubernetes.io/projected/11c1301b-ab4e-490b-8012-e973bd2b00be-kube-api-access-r94k7\") pod \"payload-pre-processing-7964457696-rnqxm\" (UID: \"11c1301b-ab4e-490b-8012-e973bd2b00be\") " pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm"
Jul 10 03:44:19.116263 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.116192    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb1e2f13-1834-43a2-8f83-786caa86eaa2-plugins-config-volume\") pod \"payload-processing-58f86c7798-ldtg2\" (UID: \"fb1e2f13-1834-43a2-8f83-786caa86eaa2\") " pod="openshift-ingress/payload-processing-58f86c7798-ldtg2"
Jul 10 03:44:19.217486 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.217403    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98q4s\" (UniqueName: \"kubernetes.io/projected/fb1e2f13-1834-43a2-8f83-786caa86eaa2-kube-api-access-98q4s\") pod \"payload-processing-58f86c7798-ldtg2\" (UID: \"fb1e2f13-1834-43a2-8f83-786caa86eaa2\") " pod="openshift-ingress/payload-processing-58f86c7798-ldtg2"
Jul 10 03:44:19.217486 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.217444    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11c1301b-ab4e-490b-8012-e973bd2b00be-plugins-config-volume\") pod \"payload-pre-processing-7964457696-rnqxm\" (UID: \"11c1301b-ab4e-490b-8012-e973bd2b00be\") " pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm"
Jul 10 03:44:19.217748 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.217498    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r94k7\" (UniqueName: \"kubernetes.io/projected/11c1301b-ab4e-490b-8012-e973bd2b00be-kube-api-access-r94k7\") pod \"payload-pre-processing-7964457696-rnqxm\" (UID: \"11c1301b-ab4e-490b-8012-e973bd2b00be\") " pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm"
Jul 10 03:44:19.217748 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.217694    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb1e2f13-1834-43a2-8f83-786caa86eaa2-plugins-config-volume\") pod \"payload-processing-58f86c7798-ldtg2\" (UID: \"fb1e2f13-1834-43a2-8f83-786caa86eaa2\") " pod="openshift-ingress/payload-processing-58f86c7798-ldtg2"
Jul 10 03:44:19.218289 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.218264    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11c1301b-ab4e-490b-8012-e973bd2b00be-plugins-config-volume\") pod \"payload-pre-processing-7964457696-rnqxm\" (UID: \"11c1301b-ab4e-490b-8012-e973bd2b00be\") " pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm"
Jul 10 03:44:19.218393 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.218361    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb1e2f13-1834-43a2-8f83-786caa86eaa2-plugins-config-volume\") pod \"payload-processing-58f86c7798-ldtg2\" (UID: \"fb1e2f13-1834-43a2-8f83-786caa86eaa2\") " pod="openshift-ingress/payload-processing-58f86c7798-ldtg2"
Jul 10 03:44:19.225725 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.225701    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98q4s\" (UniqueName: \"kubernetes.io/projected/fb1e2f13-1834-43a2-8f83-786caa86eaa2-kube-api-access-98q4s\") pod \"payload-processing-58f86c7798-ldtg2\" (UID: \"fb1e2f13-1834-43a2-8f83-786caa86eaa2\") " pod="openshift-ingress/payload-processing-58f86c7798-ldtg2"
Jul 10 03:44:19.225820 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.225773    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r94k7\" (UniqueName: \"kubernetes.io/projected/11c1301b-ab4e-490b-8012-e973bd2b00be-kube-api-access-r94k7\") pod \"payload-pre-processing-7964457696-rnqxm\" (UID: \"11c1301b-ab4e-490b-8012-e973bd2b00be\") " pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm"
Jul 10 03:44:19.368799 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.368764    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm"
Jul 10 03:44:19.386507 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.386307    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-ldtg2"
Jul 10 03:44:19.513933 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.513900    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-rnqxm"]
Jul 10 03:44:19.515423 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:44:19.515385    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11c1301b_ab4e_490b_8012_e973bd2b00be.slice/crio-8ebc1f4ad99939d4ed012efe7579a5b96552e73d55a636d136325daeaed1641a WatchSource:0}: Error finding container 8ebc1f4ad99939d4ed012efe7579a5b96552e73d55a636d136325daeaed1641a: Status 404 returned error can't find the container with id 8ebc1f4ad99939d4ed012efe7579a5b96552e73d55a636d136325daeaed1641a
Jul 10 03:44:19.546623 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:19.546601    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-ldtg2"]
Jul 10 03:44:19.547997 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:44:19.547969    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1e2f13_1834_43a2_8f83_786caa86eaa2.slice/crio-618ad282b97e881e33bf90c4c7c9739d01503fee30dd9b8934f9a9d811026cd8 WatchSource:0}: Error finding container 618ad282b97e881e33bf90c4c7c9739d01503fee30dd9b8934f9a9d811026cd8: Status 404 returned error can't find the container with id 618ad282b97e881e33bf90c4c7c9739d01503fee30dd9b8934f9a9d811026cd8
Jul 10 03:44:20.155307 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:20.155260    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-ldtg2" event={"ID":"fb1e2f13-1834-43a2-8f83-786caa86eaa2","Type":"ContainerStarted","Data":"618ad282b97e881e33bf90c4c7c9739d01503fee30dd9b8934f9a9d811026cd8"}
Jul 10 03:44:20.157264 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:20.157229    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm" event={"ID":"11c1301b-ab4e-490b-8012-e973bd2b00be","Type":"ContainerStarted","Data":"8ebc1f4ad99939d4ed012efe7579a5b96552e73d55a636d136325daeaed1641a"}
Jul 10 03:44:21.164350 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:21.164311    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm" event={"ID":"11c1301b-ab4e-490b-8012-e973bd2b00be","Type":"ContainerStarted","Data":"0dbca70d74b69dc5fee94a983a88f4c4124df09316fcffeede8b4ec5d33edb31"}
Jul 10 03:44:21.164857 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:21.164384    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm"
Jul 10 03:44:21.165818 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:21.165789    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-ldtg2" event={"ID":"fb1e2f13-1834-43a2-8f83-786caa86eaa2","Type":"ContainerStarted","Data":"25d362b7a1654eb53634373a7238ec280fd131db16ea6888dd6257923370ee98"}
Jul 10 03:44:21.165939 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:21.165842    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-ldtg2"
Jul 10 03:44:21.182174 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:21.182075    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm" podStartSLOduration=0.759438238 podStartE2EDuration="2.182057637s" totalImagesPullingTime="1.422619399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:44:21.17935085 +0000 UTC m=+722.670346739" watchObservedRunningTime="2026-07-10 03:44:21.182057637 +0000 UTC m=+722.673053511"
Jul 10 03:44:21.193616 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:21.193569    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-ldtg2" podStartSLOduration=0.805150544 podStartE2EDuration="2.193552212s" totalImagesPullingTime="1.388401668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:44:21.193549461 +0000 UTC m=+722.684545333" watchObservedRunningTime="2026-07-10 03:44:21.193552212 +0000 UTC m=+722.684548085"
Jul 10 03:44:28.242721 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:28.242686    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:44:28.936793 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:28.936756    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:44:29.037771 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:29.037740    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:44:32.172842 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:32.172809    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-ldtg2"
Jul 10 03:44:32.173200 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:32.172875    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-rnqxm"
Jul 10 03:44:36.809119 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:36.809078    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:44:39.901331 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:39.901288    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:44:58.808539 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:44:58.808502    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:45:00.137253 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:00.137215    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727585-7bh42"]
Jul 10 03:45:00.141004 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:00.140985    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-7bh42"
Jul 10 03:45:00.143306 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:00.143275    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 03:45:00.145914 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:00.145886    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727585-7bh42"]
Jul 10 03:45:00.152664 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:00.152633    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-7bh42"
Jul 10 03:45:00.311100 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:00.311072    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727585-7bh42"]
Jul 10 03:45:00.314209 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:45:00.314167    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe7272ca_08d5_4953_96bd_f9215d37d411.slice/crio-b7afa58fefe176884ad4addf94650a7bea45253b1b870589f1520137ff3ec957 WatchSource:0}: Error finding container b7afa58fefe176884ad4addf94650a7bea45253b1b870589f1520137ff3ec957: Status 404 returned error can't find the container with id b7afa58fefe176884ad4addf94650a7bea45253b1b870589f1520137ff3ec957
Jul 10 03:45:00.375793 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:00.375758    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-7bh42" event={"ID":"fe7272ca-08d5-4953-96bd-f9215d37d411","Type":"ContainerStarted","Data":"b7afa58fefe176884ad4addf94650a7bea45253b1b870589f1520137ff3ec957"}
Jul 10 03:45:04.403450 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:04.403405    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-7bh42" event={"ID":"fe7272ca-08d5-4953-96bd-f9215d37d411","Type":"ContainerStarted","Data":"0d9e7d7ed80d32760c03c6f5a8a1ef1e7dd1f3280e12fb4716eacc98a2df854d"}
Jul 10 03:45:04.416998 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:04.416939    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-7bh42" podStartSLOduration=1.177125181 podStartE2EDuration="4.416924754s" totalImagesPullingTime="3.239799573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:45:04.416242194 +0000 UTC m=+765.907238067" watchObservedRunningTime="2026-07-10 03:45:04.416924754 +0000 UTC m=+765.907920627"
Jul 10 03:45:05.410235 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:05.410196    2561 generic.go:356] "Generic (PLEG): container finished" podID="fe7272ca-08d5-4953-96bd-f9215d37d411" containerID="0d9e7d7ed80d32760c03c6f5a8a1ef1e7dd1f3280e12fb4716eacc98a2df854d" exitCode=0
Jul 10 03:45:05.410623 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:05.410287    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-7bh42" event={"ID":"fe7272ca-08d5-4953-96bd-f9215d37d411","Type":"ContainerDied","Data":"0d9e7d7ed80d32760c03c6f5a8a1ef1e7dd1f3280e12fb4716eacc98a2df854d"}
Jul 10 03:45:06.549341 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:06.549313    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-7bh42"
Jul 10 03:45:07.421992 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:07.421963    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-7bh42"
Jul 10 03:45:07.421992 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:07.421983    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-7bh42" event={"ID":"fe7272ca-08d5-4953-96bd-f9215d37d411","Type":"ContainerDied","Data":"b7afa58fefe176884ad4addf94650a7bea45253b1b870589f1520137ff3ec957"}
Jul 10 03:45:07.422210 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:07.422027    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7afa58fefe176884ad4addf94650a7bea45253b1b870589f1520137ff3ec957"
Jul 10 03:45:16.431924 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:16.431888    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:45:28.512017 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:28.511986    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:45:42.297154 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:42.297070    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:45:59.625213 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:59.625178    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:45:59.677064 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:45:59.677029    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:46:20.417871 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.417839    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6999f8c9db-jqmxt"]
Jul 10 03:46:20.418348 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.418247    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe7272ca-08d5-4953-96bd-f9215d37d411" containerName="cleanup"
Jul 10 03:46:20.418348 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.418260    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe7272ca-08d5-4953-96bd-f9215d37d411" containerName="cleanup"
Jul 10 03:46:20.418348 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.418331    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe7272ca-08d5-4953-96bd-f9215d37d411" containerName="cleanup"
Jul 10 03:46:20.421359 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.421343    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6999f8c9db-jqmxt"
Jul 10 03:46:20.430247 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.430204    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6999f8c9db-jqmxt"]
Jul 10 03:46:20.584309 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.584273    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/610204f2-9096-426d-bff7-3be22c784ec2-tls-cert\") pod \"authorino-6999f8c9db-jqmxt\" (UID: \"610204f2-9096-426d-bff7-3be22c784ec2\") " pod="kuadrant-system/authorino-6999f8c9db-jqmxt"
Jul 10 03:46:20.584482 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.584326    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kdx5\" (UniqueName: \"kubernetes.io/projected/610204f2-9096-426d-bff7-3be22c784ec2-kube-api-access-9kdx5\") pod \"authorino-6999f8c9db-jqmxt\" (UID: \"610204f2-9096-426d-bff7-3be22c784ec2\") " pod="kuadrant-system/authorino-6999f8c9db-jqmxt"
Jul 10 03:46:20.685109 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.685022    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/610204f2-9096-426d-bff7-3be22c784ec2-tls-cert\") pod \"authorino-6999f8c9db-jqmxt\" (UID: \"610204f2-9096-426d-bff7-3be22c784ec2\") " pod="kuadrant-system/authorino-6999f8c9db-jqmxt"
Jul 10 03:46:20.685109 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.685077    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kdx5\" (UniqueName: \"kubernetes.io/projected/610204f2-9096-426d-bff7-3be22c784ec2-kube-api-access-9kdx5\") pod \"authorino-6999f8c9db-jqmxt\" (UID: \"610204f2-9096-426d-bff7-3be22c784ec2\") " pod="kuadrant-system/authorino-6999f8c9db-jqmxt"
Jul 10 03:46:20.687430 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.687386    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/610204f2-9096-426d-bff7-3be22c784ec2-tls-cert\") pod \"authorino-6999f8c9db-jqmxt\" (UID: \"610204f2-9096-426d-bff7-3be22c784ec2\") " pod="kuadrant-system/authorino-6999f8c9db-jqmxt"
Jul 10 03:46:20.692504 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.692484    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kdx5\" (UniqueName: \"kubernetes.io/projected/610204f2-9096-426d-bff7-3be22c784ec2-kube-api-access-9kdx5\") pod \"authorino-6999f8c9db-jqmxt\" (UID: \"610204f2-9096-426d-bff7-3be22c784ec2\") " pod="kuadrant-system/authorino-6999f8c9db-jqmxt"
Jul 10 03:46:20.731911 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.731869    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6999f8c9db-jqmxt"
Jul 10 03:46:20.813693 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.813642    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:46:20.868673 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.868625    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6999f8c9db-jqmxt"]
Jul 10 03:46:20.869976 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:46:20.869950    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod610204f2_9096_426d_bff7_3be22c784ec2.slice/crio-c6cbc405458173d93eca273a6414843e068941e1b2736f3a927020e41dfcb2ad WatchSource:0}: Error finding container c6cbc405458173d93eca273a6414843e068941e1b2736f3a927020e41dfcb2ad: Status 404 returned error can't find the container with id c6cbc405458173d93eca273a6414843e068941e1b2736f3a927020e41dfcb2ad
Jul 10 03:46:20.912478 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:20.912447    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:46:21.815901 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:21.815859    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6999f8c9db-jqmxt" event={"ID":"610204f2-9096-426d-bff7-3be22c784ec2","Type":"ContainerStarted","Data":"436c94c3df6e03b4fb617c157e5449a5e2155fc13161dff8e8bde74c82a89b68"}
Jul 10 03:46:21.816368 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:21.815912    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6999f8c9db-jqmxt" event={"ID":"610204f2-9096-426d-bff7-3be22c784ec2","Type":"ContainerStarted","Data":"c6cbc405458173d93eca273a6414843e068941e1b2736f3a927020e41dfcb2ad"}
Jul 10 03:46:21.831949 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:21.831885    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6999f8c9db-jqmxt" podStartSLOduration=1.449644584 podStartE2EDuration="1.831863967s" totalImagesPullingTime="382.219383ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:46:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:46:21.828943247 +0000 UTC m=+843.319939117" watchObservedRunningTime="2026-07-10 03:46:21.831863967 +0000 UTC m=+843.322859840"
Jul 10 03:46:21.861669 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:21.858947    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86f5f5c6c-7pmwb"]
Jul 10 03:46:21.861669 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:21.861431    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-86f5f5c6c-7pmwb" podUID="2f24b82c-e9f8-4ab9-bc03-47c5849bda12" containerName="authorino" containerID="cri-o://e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c" gracePeriod=30
Jul 10 03:46:22.040944 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.040845    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:46:22.136524 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.136494    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:46:22.144368 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.144346    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86f5f5c6c-7pmwb"
Jul 10 03:46:22.299702 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.299667    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2f24b82c-e9f8-4ab9-bc03-47c5849bda12-tls-cert\") pod \"2f24b82c-e9f8-4ab9-bc03-47c5849bda12\" (UID: \"2f24b82c-e9f8-4ab9-bc03-47c5849bda12\") "
Jul 10 03:46:22.299865 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.299767    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxjpd\" (UniqueName: \"kubernetes.io/projected/2f24b82c-e9f8-4ab9-bc03-47c5849bda12-kube-api-access-sxjpd\") pod \"2f24b82c-e9f8-4ab9-bc03-47c5849bda12\" (UID: \"2f24b82c-e9f8-4ab9-bc03-47c5849bda12\") "
Jul 10 03:46:22.301905 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.301868    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f24b82c-e9f8-4ab9-bc03-47c5849bda12-kube-api-access-sxjpd" (OuterVolumeSpecName: "kube-api-access-sxjpd") pod "2f24b82c-e9f8-4ab9-bc03-47c5849bda12" (UID: "2f24b82c-e9f8-4ab9-bc03-47c5849bda12"). InnerVolumeSpecName "kube-api-access-sxjpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:46:22.309871 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.309843    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f24b82c-e9f8-4ab9-bc03-47c5849bda12-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2f24b82c-e9f8-4ab9-bc03-47c5849bda12" (UID: "2f24b82c-e9f8-4ab9-bc03-47c5849bda12"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:46:22.401438 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.401357    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxjpd\" (UniqueName: \"kubernetes.io/projected/2f24b82c-e9f8-4ab9-bc03-47c5849bda12-kube-api-access-sxjpd\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:46:22.401438 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.401387    2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2f24b82c-e9f8-4ab9-bc03-47c5849bda12-tls-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:46:22.823805 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.823759    2561 generic.go:356] "Generic (PLEG): container finished" podID="2f24b82c-e9f8-4ab9-bc03-47c5849bda12" containerID="e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c" exitCode=0
Jul 10 03:46:22.824234 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.823833    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86f5f5c6c-7pmwb"
Jul 10 03:46:22.824234 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.823854    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86f5f5c6c-7pmwb" event={"ID":"2f24b82c-e9f8-4ab9-bc03-47c5849bda12","Type":"ContainerDied","Data":"e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c"}
Jul 10 03:46:22.824234 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.823910    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86f5f5c6c-7pmwb" event={"ID":"2f24b82c-e9f8-4ab9-bc03-47c5849bda12","Type":"ContainerDied","Data":"9cc88143e5544b77852814158253a1e772b255fa5060f7eb813eb69742bbcf8b"}
Jul 10 03:46:22.824234 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.823938    2561 scope.go:117] "RemoveContainer" containerID="e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c"
Jul 10 03:46:22.834283 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.834265    2561 scope.go:117] "RemoveContainer" containerID="e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c"
Jul 10 03:46:22.834555 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:46:22.834537    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c\": container with ID starting with e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c not found: ID does not exist" containerID="e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c"
Jul 10 03:46:22.834602 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.834564    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c"} err="failed to get container status \"e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c\": rpc error: code = NotFound desc = could not find container \"e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c\": container with ID starting with e1cabebb699ee85fe0744a34d191a4ef0cea250e14df9c34530f6f1d30ff975c not found: ID does not exist"
Jul 10 03:46:22.845743 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.845720    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86f5f5c6c-7pmwb"]
Jul 10 03:46:22.851437 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.851404    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-86f5f5c6c-7pmwb"]
Jul 10 03:46:22.995544 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:22.995470    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f24b82c-e9f8-4ab9-bc03-47c5849bda12" path="/var/lib/kubelet/pods/2f24b82c-e9f8-4ab9-bc03-47c5849bda12/volumes"
Jul 10 03:46:47.793553 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:47.793518    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:46:51.586079 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:46:51.586034    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:47:19.021581 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:19.021552    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:47:19.022851 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:19.022826    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:47:32.881866 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:32.881831    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:47:32.984746 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:32.984712    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f864f4b59-m79h4"]
Jul 10 03:47:32.984955 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:32.984913    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f864f4b59-m79h4" podUID="0863097f-adfe-46e0-a4df-462d9161934b" containerName="manager" containerID="cri-o://e54401e873bb6b01b40f1aaf9ab9f8fcb67abe2790f8593639f647f1ca27d8f4" gracePeriod=30
Jul 10 03:47:33.213742 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:33.213712    2561 generic.go:356] "Generic (PLEG): container finished" podID="0863097f-adfe-46e0-a4df-462d9161934b" containerID="e54401e873bb6b01b40f1aaf9ab9f8fcb67abe2790f8593639f647f1ca27d8f4" exitCode=0
Jul 10 03:47:33.213870 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:33.213780    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f864f4b59-m79h4" event={"ID":"0863097f-adfe-46e0-a4df-462d9161934b","Type":"ContainerDied","Data":"e54401e873bb6b01b40f1aaf9ab9f8fcb67abe2790f8593639f647f1ca27d8f4"}
Jul 10 03:47:33.233547 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:33.233518    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f864f4b59-m79h4"
Jul 10 03:47:33.317290 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:33.317260    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pw8gc\" (UniqueName: \"kubernetes.io/projected/0863097f-adfe-46e0-a4df-462d9161934b-kube-api-access-pw8gc\") pod \"0863097f-adfe-46e0-a4df-462d9161934b\" (UID: \"0863097f-adfe-46e0-a4df-462d9161934b\") "
Jul 10 03:47:33.317443 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:33.317305    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0863097f-adfe-46e0-a4df-462d9161934b-webhook-cert\") pod \"0863097f-adfe-46e0-a4df-462d9161934b\" (UID: \"0863097f-adfe-46e0-a4df-462d9161934b\") "
Jul 10 03:47:33.319330 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:33.319306    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0863097f-adfe-46e0-a4df-462d9161934b-kube-api-access-pw8gc" (OuterVolumeSpecName: "kube-api-access-pw8gc") pod "0863097f-adfe-46e0-a4df-462d9161934b" (UID: "0863097f-adfe-46e0-a4df-462d9161934b"). InnerVolumeSpecName "kube-api-access-pw8gc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:47:33.319409 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:33.319342    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0863097f-adfe-46e0-a4df-462d9161934b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0863097f-adfe-46e0-a4df-462d9161934b" (UID: "0863097f-adfe-46e0-a4df-462d9161934b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:47:33.418055 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:33.417976    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw8gc\" (UniqueName: \"kubernetes.io/projected/0863097f-adfe-46e0-a4df-462d9161934b-kube-api-access-pw8gc\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:47:33.418055 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:33.418003    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0863097f-adfe-46e0-a4df-462d9161934b-webhook-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 03:47:34.220539 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:34.220504    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f864f4b59-m79h4"
Jul 10 03:47:34.220974 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:34.220540    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f864f4b59-m79h4" event={"ID":"0863097f-adfe-46e0-a4df-462d9161934b","Type":"ContainerDied","Data":"d85a4fa60984e1867c2858500f23e5ef8e07e859ac3829f3d813119838dd4d5f"}
Jul 10 03:47:34.220974 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:34.220598    2561 scope.go:117] "RemoveContainer" containerID="e54401e873bb6b01b40f1aaf9ab9f8fcb67abe2790f8593639f647f1ca27d8f4"
Jul 10 03:47:34.241409 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:34.241386    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f864f4b59-m79h4"]
Jul 10 03:47:34.244741 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:34.244717    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f864f4b59-m79h4"]
Jul 10 03:47:34.930814 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:34.930784    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:47:34.993568 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:34.993539    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0863097f-adfe-46e0-a4df-462d9161934b" path="/var/lib/kubelet/pods/0863097f-adfe-46e0-a4df-462d9161934b/volumes"
Jul 10 03:47:35.050542 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.050505    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f864f4b59-9h9qh"]
Jul 10 03:47:35.050913 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.050899    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0863097f-adfe-46e0-a4df-462d9161934b" containerName="manager"
Jul 10 03:47:35.050969 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.050915    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0863097f-adfe-46e0-a4df-462d9161934b" containerName="manager"
Jul 10 03:47:35.050969 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.050935    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f24b82c-e9f8-4ab9-bc03-47c5849bda12" containerName="authorino"
Jul 10 03:47:35.050969 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.050940    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f24b82c-e9f8-4ab9-bc03-47c5849bda12" containerName="authorino"
Jul 10 03:47:35.051063 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.050992    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0863097f-adfe-46e0-a4df-462d9161934b" containerName="manager"
Jul 10 03:47:35.051063 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.051003    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f24b82c-e9f8-4ab9-bc03-47c5849bda12" containerName="authorino"
Jul 10 03:47:35.055287 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.055270    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f864f4b59-9h9qh"
Jul 10 03:47:35.058289 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.058268    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 10 03:47:35.058289 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.058271    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 10 03:47:35.058496 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.058272    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ngm4n\""
Jul 10 03:47:35.061385 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.061362    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f864f4b59-9h9qh"]
Jul 10 03:47:35.133872 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.133832    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qwgc\" (UniqueName: \"kubernetes.io/projected/dc45a6d4-dac9-497f-91b9-873e433b0dde-kube-api-access-4qwgc\") pod \"maas-controller-6f864f4b59-9h9qh\" (UID: \"dc45a6d4-dac9-497f-91b9-873e433b0dde\") " pod="opendatahub/maas-controller-6f864f4b59-9h9qh"
Jul 10 03:47:35.134058 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.133952    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dc45a6d4-dac9-497f-91b9-873e433b0dde-webhook-cert\") pod \"maas-controller-6f864f4b59-9h9qh\" (UID: \"dc45a6d4-dac9-497f-91b9-873e433b0dde\") " pod="opendatahub/maas-controller-6f864f4b59-9h9qh"
Jul 10 03:47:35.235341 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.235246    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dc45a6d4-dac9-497f-91b9-873e433b0dde-webhook-cert\") pod \"maas-controller-6f864f4b59-9h9qh\" (UID: \"dc45a6d4-dac9-497f-91b9-873e433b0dde\") " pod="opendatahub/maas-controller-6f864f4b59-9h9qh"
Jul 10 03:47:35.235341 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.235321    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qwgc\" (UniqueName: \"kubernetes.io/projected/dc45a6d4-dac9-497f-91b9-873e433b0dde-kube-api-access-4qwgc\") pod \"maas-controller-6f864f4b59-9h9qh\" (UID: \"dc45a6d4-dac9-497f-91b9-873e433b0dde\") " pod="opendatahub/maas-controller-6f864f4b59-9h9qh"
Jul 10 03:47:35.237544 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.237523    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dc45a6d4-dac9-497f-91b9-873e433b0dde-webhook-cert\") pod \"maas-controller-6f864f4b59-9h9qh\" (UID: \"dc45a6d4-dac9-497f-91b9-873e433b0dde\") " pod="opendatahub/maas-controller-6f864f4b59-9h9qh"
Jul 10 03:47:35.242175 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.242153    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qwgc\" (UniqueName: \"kubernetes.io/projected/dc45a6d4-dac9-497f-91b9-873e433b0dde-kube-api-access-4qwgc\") pod \"maas-controller-6f864f4b59-9h9qh\" (UID: \"dc45a6d4-dac9-497f-91b9-873e433b0dde\") " pod="opendatahub/maas-controller-6f864f4b59-9h9qh"
Jul 10 03:47:35.366432 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.366398    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f864f4b59-9h9qh"
Jul 10 03:47:35.496154 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:35.496130    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f864f4b59-9h9qh"]
Jul 10 03:47:35.497480 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:47:35.497453    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc45a6d4_dac9_497f_91b9_873e433b0dde.slice/crio-a3e2ad061de0908540289b53a6a3598db74324cf44e804567a98e0d9aa545cfc WatchSource:0}: Error finding container a3e2ad061de0908540289b53a6a3598db74324cf44e804567a98e0d9aa545cfc: Status 404 returned error can't find the container with id a3e2ad061de0908540289b53a6a3598db74324cf44e804567a98e0d9aa545cfc
Jul 10 03:47:36.234421 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:36.234327    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f864f4b59-9h9qh" event={"ID":"dc45a6d4-dac9-497f-91b9-873e433b0dde","Type":"ContainerStarted","Data":"b3aa41df1345cd60738c81f2435430f40a8b3a52ab63812c50e23cbeb3ee51d6"}
Jul 10 03:47:36.234421 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:36.234396    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f864f4b59-9h9qh"
Jul 10 03:47:36.234421 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:36.234419    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f864f4b59-9h9qh" event={"ID":"dc45a6d4-dac9-497f-91b9-873e433b0dde","Type":"ContainerStarted","Data":"a3e2ad061de0908540289b53a6a3598db74324cf44e804567a98e0d9aa545cfc"}
Jul 10 03:47:36.253228 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:36.253172    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f864f4b59-9h9qh" podStartSLOduration=0.821220314 podStartE2EDuration="1.25315901s" totalImagesPullingTime="431.938696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:47:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:47:36.251776469 +0000 UTC m=+917.742772340" watchObservedRunningTime="2026-07-10 03:47:36.25315901 +0000 UTC m=+917.744154881"
Jul 10 03:47:47.242462 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:47:47.242430    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f864f4b59-9h9qh"
Jul 10 03:48:15.791139 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:48:15.791085    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:48:15.897199 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:48:15.897145    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:48:16.040577 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:48:16.040544    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:48:21.596292 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:48:21.596258    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:48:21.706524 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:48:21.706484    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:48:21.845797 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:48:21.845767    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:48:21.991735 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:48:21.991611    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:48:27.866564 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:48:27.866526    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:48:27.966597 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:48:27.966553    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:48:45.672990 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:48:45.672955    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:48:45.724050 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:48:45.724016    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:50:12.837295 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:50:12.837212    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:52:19.057738 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:52:19.057708    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:52:19.068665 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:52:19.068591    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:53:17.591927 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:53:17.591850    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:53:53.721772 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:53:53.721736    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:53:53.775542 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:53:53.775508    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:55:56.596170 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:55:56.596123    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:57:17.598850 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:57:17.598812    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:57:19.101195 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:57:19.101159    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:57:19.107898 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:57:19.107875    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 03:57:28.298991 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:57:28.298955    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:15.517367 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:15.517290    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:25.313552 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:25.313513    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:33.585401 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:33.585362    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:38.753867 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:38.753830    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:41.748223 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:41.748180    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:41.848080 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:41.848048    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:42.225008 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:42.224973    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:47.916890 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:47.916852    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:48.024078 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.024049    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:48.102237 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.102201    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"]
Jul 10 03:59:48.105710 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.105685    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.108019 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.108003    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-11393920-openshift-default-dockercfg-zjxrt\""
Jul 10 03:59:48.108192 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.108174    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 03:59:48.124730 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.124705    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"]
Jul 10 03:59:48.128912 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.128892    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:48.222122 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.222051    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:48.276349 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.276316    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-envoy\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.276509 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.276352    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-podinfo\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.276509 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.276382    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-workload-socket\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.276509 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.276404    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istiod-ca-cert\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.276509 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.276430    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-credential-socket\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.276509 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.276453    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq9dg\" (UniqueName: \"kubernetes.io/projected/1fe65fe7-1b0d-462f-9d79-6822befae7cc-kube-api-access-vq9dg\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.276509 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.276489    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-workload-certs\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.276746 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.276537    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-data\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.276746 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.276580    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-token\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.377991 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.377957    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-workload-certs\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.378183 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.378001    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-data\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.378183 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.378045    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-token\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.378183 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.378101    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-envoy\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.378183 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.378142    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-podinfo\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.378183 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.378178    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-workload-socket\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.378459 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.378205    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istiod-ca-cert\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.378459 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.378240    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-credential-socket\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.378459 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.378270    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq9dg\" (UniqueName: \"kubernetes.io/projected/1fe65fe7-1b0d-462f-9d79-6822befae7cc-kube-api-access-vq9dg\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.378623 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.378493    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-data\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.378721 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.378683    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-workload-socket\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.378778 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.378728    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-workload-certs\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.378778 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.378750    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-credential-socket\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.379030 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.379009    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istiod-ca-cert\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.380481 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.380459    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-envoy\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.380689 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.380670    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-podinfo\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.385896 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.385868    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-token\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.386320 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.386293    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq9dg\" (UniqueName: \"kubernetes.io/projected/1fe65fe7-1b0d-462f-9d79-6822befae7cc-kube-api-access-vq9dg\") pod \"e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") " pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.416938 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.416914    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:48.555457 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.555429    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"]
Jul 10 03:59:48.557490 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:59:48.557455    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe65fe7_1b0d_462f_9d79_6822befae7cc.slice/crio-a75aa5da30f8783765c860cb1b09cc8aaccc935d0ddaed9b673d6bcc8ab12550 WatchSource:0}: Error finding container a75aa5da30f8783765c860cb1b09cc8aaccc935d0ddaed9b673d6bcc8ab12550: Status 404 returned error can't find the container with id a75aa5da30f8783765c860cb1b09cc8aaccc935d0ddaed9b673d6bcc8ab12550
Jul 10 03:59:48.559511 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:48.559494    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 03:59:49.221520 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.221472    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz" event={"ID":"1fe65fe7-1b0d-462f-9d79-6822befae7cc","Type":"ContainerStarted","Data":"a75aa5da30f8783765c860cb1b09cc8aaccc935d0ddaed9b673d6bcc8ab12550"}
Jul 10 03:59:49.250697 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.250628    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:49.346944 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.346905    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:49.442942 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.442913    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"]
Jul 10 03:59:49.448207 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.448187    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:49.450806 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.450633    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-11393920-serving-cert\""
Jul 10 03:59:49.450806 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.450725    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 03:59:49.452881 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.451812    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 03:59:49.452881 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.452339    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6g9gw\""
Jul 10 03:59:49.456727 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.456687    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"]
Jul 10 03:59:49.472183 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.472085    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:49.590522 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.590461    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00186fae-0c69-47e4-bde5-913f436ff2d7-openshift-service-ca\") pod \"maas-api-e2e-ait-11393920-768777dd74-7p6b8\" (UID: \"00186fae-0c69-47e4-bde5-913f436ff2d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:49.590522 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.590518    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00186fae-0c69-47e4-bde5-913f436ff2d7-maas-api-tls\") pod \"maas-api-e2e-ait-11393920-768777dd74-7p6b8\" (UID: \"00186fae-0c69-47e4-bde5-913f436ff2d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:49.590810 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.590609    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvsz7\" (UniqueName: \"kubernetes.io/projected/00186fae-0c69-47e4-bde5-913f436ff2d7-kube-api-access-qvsz7\") pod \"maas-api-e2e-ait-11393920-768777dd74-7p6b8\" (UID: \"00186fae-0c69-47e4-bde5-913f436ff2d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:49.678538 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.678485    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:49.691692 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.691633    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00186fae-0c69-47e4-bde5-913f436ff2d7-openshift-service-ca\") pod \"maas-api-e2e-ait-11393920-768777dd74-7p6b8\" (UID: \"00186fae-0c69-47e4-bde5-913f436ff2d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:49.691881 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.691712    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00186fae-0c69-47e4-bde5-913f436ff2d7-maas-api-tls\") pod \"maas-api-e2e-ait-11393920-768777dd74-7p6b8\" (UID: \"00186fae-0c69-47e4-bde5-913f436ff2d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:49.691881 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.691757    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvsz7\" (UniqueName: \"kubernetes.io/projected/00186fae-0c69-47e4-bde5-913f436ff2d7-kube-api-access-qvsz7\") pod \"maas-api-e2e-ait-11393920-768777dd74-7p6b8\" (UID: \"00186fae-0c69-47e4-bde5-913f436ff2d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:49.692408 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.692381    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00186fae-0c69-47e4-bde5-913f436ff2d7-openshift-service-ca\") pod \"maas-api-e2e-ait-11393920-768777dd74-7p6b8\" (UID: \"00186fae-0c69-47e4-bde5-913f436ff2d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:49.694820 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.694796    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00186fae-0c69-47e4-bde5-913f436ff2d7-maas-api-tls\") pod \"maas-api-e2e-ait-11393920-768777dd74-7p6b8\" (UID: \"00186fae-0c69-47e4-bde5-913f436ff2d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:49.704540 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.704518    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvsz7\" (UniqueName: \"kubernetes.io/projected/00186fae-0c69-47e4-bde5-913f436ff2d7-kube-api-access-qvsz7\") pod \"maas-api-e2e-ait-11393920-768777dd74-7p6b8\" (UID: \"00186fae-0c69-47e4-bde5-913f436ff2d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:49.771680 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.771586    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:49.980784 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:49.980750    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"]
Jul 10 03:59:49.984189 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:59:49.984161    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00186fae_0c69_47e4_bde5_913f436ff2d7.slice/crio-60af13e6ac6743184467a615ea9097f0fbcbb2f9598546d4dc2a24209e4fc067 WatchSource:0}: Error finding container 60af13e6ac6743184467a615ea9097f0fbcbb2f9598546d4dc2a24209e4fc067: Status 404 returned error can't find the container with id 60af13e6ac6743184467a615ea9097f0fbcbb2f9598546d4dc2a24209e4fc067
Jul 10 03:59:50.087332 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:50.087293    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:50.189545 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:50.189513    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:50.228910 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:50.228878    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8" event={"ID":"00186fae-0c69-47e4-bde5-913f436ff2d7","Type":"ContainerStarted","Data":"60af13e6ac6743184467a615ea9097f0fbcbb2f9598546d4dc2a24209e4fc067"}
Jul 10 03:59:51.674158 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:51.674119    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 03:59:52.099855 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:52.099818    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 03:59:52.099966 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:52.099891    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 03:59:52.099966 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:52.099918    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 03:59:52.243016 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:52.242982    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz" event={"ID":"1fe65fe7-1b0d-462f-9d79-6822befae7cc","Type":"ContainerStarted","Data":"fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6"}
Jul 10 03:59:52.244816 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:52.244783    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8" event={"ID":"00186fae-0c69-47e4-bde5-913f436ff2d7","Type":"ContainerStarted","Data":"c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755"}
Jul 10 03:59:52.244968 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:52.244843    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:52.264107 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:52.264056    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz" podStartSLOduration=0.724192303 podStartE2EDuration="4.264037904s" totalImagesPullingTime="3.539845601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:59:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:59:52.260791392 +0000 UTC m=+1653.751787265" watchObservedRunningTime="2026-07-10 03:59:52.264037904 +0000 UTC m=+1653.755033778"
Jul 10 03:59:52.275552 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:52.275500    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8" podStartSLOduration=1.162598051 podStartE2EDuration="3.275482977s" totalImagesPullingTime="2.112884926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:59:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:59:52.275277927 +0000 UTC m=+1653.766273801" watchObservedRunningTime="2026-07-10 03:59:52.275482977 +0000 UTC m=+1653.766478849"
Jul 10 03:59:52.417955 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:52.417880    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:53.422224 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:53.422191    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:54.259070 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:54.259028    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:54.282600 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:54.278423    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 03:59:54.364695 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:54.364659    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"]
Jul 10 03:59:54.368455 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:54.368436    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"
Jul 10 03:59:54.370605 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:54.370583    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 03:59:54.374801 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:54.374781    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"]
Jul 10 03:59:54.535297 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:54.535210    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b303a4c5-c22d-4751-9713-4e297bce4b66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-11393920-xqk47\" (UID: \"b303a4c5-c22d-4751-9713-4e297bce4b66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"
Jul 10 03:59:54.635806 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:54.635757    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b303a4c5-c22d-4751-9713-4e297bce4b66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-11393920-xqk47\" (UID: \"b303a4c5-c22d-4751-9713-4e297bce4b66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"
Jul 10 03:59:54.636544 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:54.636508    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b303a4c5-c22d-4751-9713-4e297bce4b66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-11393920-xqk47\" (UID: \"b303a4c5-c22d-4751-9713-4e297bce4b66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"
Jul 10 03:59:54.699833 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:54.699780    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"
Jul 10 03:59:54.863488 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:54.863460    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"]
Jul 10 03:59:54.865410 ip-10-0-140-167 kubenswrapper[2561]: W0710 03:59:54.865381    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb303a4c5_c22d_4751_9713_4e297bce4b66.slice/crio-cf206a4e67554c64d6c35616d82a6ab3a589b66c3eac34c7bae076a476aba244 WatchSource:0}: Error finding container cf206a4e67554c64d6c35616d82a6ab3a589b66c3eac34c7bae076a476aba244: Status 404 returned error can't find the container with id cf206a4e67554c64d6c35616d82a6ab3a589b66c3eac34c7bae076a476aba244
Jul 10 03:59:55.265296 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:55.265260    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47" event={"ID":"b303a4c5-c22d-4751-9713-4e297bce4b66","Type":"ContainerStarted","Data":"cf206a4e67554c64d6c35616d82a6ab3a589b66c3eac34c7bae076a476aba244"}
Jul 10 03:59:56.271785 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:56.271751    2561 generic.go:356] "Generic (PLEG): container finished" podID="b303a4c5-c22d-4751-9713-4e297bce4b66" containerID="91a777d48d4275e224eb0b7b7e26fe29c9979d00e88f1bb507563959b316d7fd" exitCode=7
Jul 10 03:59:56.272184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:56.271822    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47" event={"ID":"b303a4c5-c22d-4751-9713-4e297bce4b66","Type":"ContainerDied","Data":"91a777d48d4275e224eb0b7b7e26fe29c9979d00e88f1bb507563959b316d7fd"}
Jul 10 03:59:56.272184 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:56.272022    2561 scope.go:117] "RemoveContainer" containerID="91a777d48d4275e224eb0b7b7e26fe29c9979d00e88f1bb507563959b316d7fd"
Jul 10 03:59:57.291690 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:57.290003    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47" event={"ID":"b303a4c5-c22d-4751-9713-4e297bce4b66","Type":"ContainerStarted","Data":"164b66385160128b5b703e2d3739ae851e1f813111e1ffa1e837e08cef6535fa"}
Jul 10 03:59:57.304982 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:57.304900    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47" podStartSLOduration=3.304878341 podStartE2EDuration="3.304878341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:59:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:59:57.304402318 +0000 UTC m=+1658.795398193" watchObservedRunningTime="2026-07-10 03:59:57.304878341 +0000 UTC m=+1658.795874218"
Jul 10 03:59:58.255982 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:58.255954    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 03:59:58.307343 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:58.306056    2561 generic.go:356] "Generic (PLEG): container finished" podID="b303a4c5-c22d-4751-9713-4e297bce4b66" containerID="164b66385160128b5b703e2d3739ae851e1f813111e1ffa1e837e08cef6535fa" exitCode=7
Jul 10 03:59:58.307343 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:58.306213    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47" event={"ID":"b303a4c5-c22d-4751-9713-4e297bce4b66","Type":"ContainerDied","Data":"164b66385160128b5b703e2d3739ae851e1f813111e1ffa1e837e08cef6535fa"}
Jul 10 03:59:58.307343 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:58.306279    2561 scope.go:117] "RemoveContainer" containerID="91a777d48d4275e224eb0b7b7e26fe29c9979d00e88f1bb507563959b316d7fd"
Jul 10 03:59:58.307343 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:58.306725    2561 scope.go:117] "RemoveContainer" containerID="164b66385160128b5b703e2d3739ae851e1f813111e1ffa1e837e08cef6535fa"
Jul 10 03:59:58.307343 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:59:58.306977    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-11393920-xqk47_odh-ai-gateway-infra(b303a4c5-c22d-4751-9713-4e297bce4b66)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47" podUID="b303a4c5-c22d-4751-9713-4e297bce4b66"
Jul 10 03:59:59.314059 ip-10-0-140-167 kubenswrapper[2561]: I0710 03:59:59.314025    2561 scope.go:117] "RemoveContainer" containerID="164b66385160128b5b703e2d3739ae851e1f813111e1ffa1e837e08cef6535fa"
Jul 10 03:59:59.314428 ip-10-0-140-167 kubenswrapper[2561]: E0710 03:59:59.314278    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-11393920-xqk47_odh-ai-gateway-infra(b303a4c5-c22d-4751-9713-4e297bce4b66)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47" podUID="b303a4c5-c22d-4751-9713-4e297bce4b66"
Jul 10 04:00:00.126451 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:00.126417    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g"]
Jul 10 04:00:00.131986 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:00.131963    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727600-qjvxv"]
Jul 10 04:00:00.132146 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:00.132106    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g"
Jul 10 04:00:00.136179 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:00.136154    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727600-qjvxv"
Jul 10 04:00:00.139152 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:00.139021    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g"]
Jul 10 04:00:00.145482 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:00.145458    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727600-qjvxv"]
Jul 10 04:00:00.148804 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:00.148777    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g"
Jul 10 04:00:00.156697 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:00.156679    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727600-qjvxv"
Jul 10 04:00:00.326575 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:00.326548    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727600-qjvxv"]
Jul 10 04:00:00.327979 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:00:00.327952    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c68fd1_41e4_460b_9b40_1c1d66d08bce.slice/crio-538132ef857b5ef3e42b780f495e105eadde487cb9126443e30395539ed3b401 WatchSource:0}: Error finding container 538132ef857b5ef3e42b780f495e105eadde487cb9126443e30395539ed3b401: Status 404 returned error can't find the container with id 538132ef857b5ef3e42b780f495e105eadde487cb9126443e30395539ed3b401
Jul 10 04:00:00.512622 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:00.512592    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g"]
Jul 10 04:00:00.513969 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:00:00.513941    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f0e816_96a6_4cd0_a90b_75cb9fcb90df.slice/crio-67a8d4782695c5727814f803b3c1dff28edcfb994a3e0c6973c5885c3da1a380 WatchSource:0}: Error finding container 67a8d4782695c5727814f803b3c1dff28edcfb994a3e0c6973c5885c3da1a380: Status 404 returned error can't find the container with id 67a8d4782695c5727814f803b3c1dff28edcfb994a3e0c6973c5885c3da1a380
Jul 10 04:00:01.327208 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:01.327172    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727600-qjvxv" event={"ID":"d1c68fd1-41e4-460b-9b40-1c1d66d08bce","Type":"ContainerStarted","Data":"dbfa8c3259f9ed6e077ddb3999e59e2a925761b0aa94b3b4d78a2362f43c3998"}
Jul 10 04:00:01.327710 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:01.327228    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727600-qjvxv" event={"ID":"d1c68fd1-41e4-460b-9b40-1c1d66d08bce","Type":"ContainerStarted","Data":"538132ef857b5ef3e42b780f495e105eadde487cb9126443e30395539ed3b401"}
Jul 10 04:00:01.328738 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:01.328717    2561 generic.go:356] "Generic (PLEG): container finished" podID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerID="fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6" exitCode=0
Jul 10 04:00:01.328805 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:01.328786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz" event={"ID":"1fe65fe7-1b0d-462f-9d79-6822befae7cc","Type":"ContainerDied","Data":"fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6"}
Jul 10 04:00:01.329171 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:01.329155    2561 scope.go:117] "RemoveContainer" containerID="fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6"
Jul 10 04:00:01.330438 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:01.330415    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g" event={"ID":"06f0e816-96a6-4cd0-a90b-75cb9fcb90df","Type":"ContainerStarted","Data":"8de3b925fd0bd2378efa12d41a04a7eeab7c74c985ed09d895a1d647087323d1"}
Jul 10 04:00:01.330723 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:01.330459    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g" event={"ID":"06f0e816-96a6-4cd0-a90b-75cb9fcb90df","Type":"ContainerStarted","Data":"67a8d4782695c5727814f803b3c1dff28edcfb994a3e0c6973c5885c3da1a380"}
Jul 10 04:00:01.331403 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:01.331375    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:00:01.331486 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:01.331441    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:00:01.331486 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:01.331475    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:00:01.343611 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:01.343563    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727600-qjvxv" podStartSLOduration=1.343546949 podStartE2EDuration="1.343546949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:00:01.340784393 +0000 UTC m=+1662.831780269" watchObservedRunningTime="2026-07-10 04:00:01.343546949 +0000 UTC m=+1662.834542858"
Jul 10 04:00:01.373361 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:01.373296    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g" podStartSLOduration=1.373270714 podStartE2EDuration="1.373270714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:00:01.369918798 +0000 UTC m=+1662.860914672" watchObservedRunningTime="2026-07-10 04:00:01.373270714 +0000 UTC m=+1662.864266588"
Jul 10 04:00:02.337243 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:02.337212    2561 generic.go:356] "Generic (PLEG): container finished" podID="d1c68fd1-41e4-460b-9b40-1c1d66d08bce" containerID="dbfa8c3259f9ed6e077ddb3999e59e2a925761b0aa94b3b4d78a2362f43c3998" exitCode=0
Jul 10 04:00:02.337732 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:02.337297    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727600-qjvxv" event={"ID":"d1c68fd1-41e4-460b-9b40-1c1d66d08bce","Type":"ContainerDied","Data":"dbfa8c3259f9ed6e077ddb3999e59e2a925761b0aa94b3b4d78a2362f43c3998"}
Jul 10 04:00:02.339180 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:02.339158    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz" event={"ID":"1fe65fe7-1b0d-462f-9d79-6822befae7cc","Type":"ContainerStarted","Data":"550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9"}
Jul 10 04:00:02.340778 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:02.340756    2561 generic.go:356] "Generic (PLEG): container finished" podID="06f0e816-96a6-4cd0-a90b-75cb9fcb90df" containerID="8de3b925fd0bd2378efa12d41a04a7eeab7c74c985ed09d895a1d647087323d1" exitCode=0
Jul 10 04:00:02.340860 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:02.340812    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g" event={"ID":"06f0e816-96a6-4cd0-a90b-75cb9fcb90df","Type":"ContainerDied","Data":"8de3b925fd0bd2378efa12d41a04a7eeab7c74c985ed09d895a1d647087323d1"}
Jul 10 04:00:02.417893 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:02.417846    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 04:00:02.418535 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:02.418513    2561 patch_prober.go:28] interesting pod/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 10 04:00:02.418592 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:02.418563    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz" podUID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 10 04:00:03.417450 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:03.417416    2561 patch_prober.go:28] interesting pod/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 10 04:00:03.417940 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:03.417466    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz" podUID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 10 04:00:03.539475 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:03.539448    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727600-qjvxv"
Jul 10 04:00:03.565385 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:03.565359    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g"
Jul 10 04:00:04.352613 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:04.352575    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g"
Jul 10 04:00:04.352814 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:04.352629    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g" event={"ID":"06f0e816-96a6-4cd0-a90b-75cb9fcb90df","Type":"ContainerDied","Data":"67a8d4782695c5727814f803b3c1dff28edcfb994a3e0c6973c5885c3da1a380"}
Jul 10 04:00:04.352814 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:04.352697    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67a8d4782695c5727814f803b3c1dff28edcfb994a3e0c6973c5885c3da1a380"
Jul 10 04:00:04.353900 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:04.353881    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727600-qjvxv" event={"ID":"d1c68fd1-41e4-460b-9b40-1c1d66d08bce","Type":"ContainerDied","Data":"538132ef857b5ef3e42b780f495e105eadde487cb9126443e30395539ed3b401"}
Jul 10 04:00:04.353900 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:04.353900    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727600-qjvxv"
Jul 10 04:00:04.354037 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:04.353918    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="538132ef857b5ef3e42b780f495e105eadde487cb9126443e30395539ed3b401"
Jul 10 04:00:04.417733 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:04.417699    2561 patch_prober.go:28] interesting pod/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 10 04:00:04.418119 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:04.417764    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz" podUID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 10 04:00:05.417981 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:05.417940    2561 patch_prober.go:28] interesting pod/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 10 04:00:05.418394 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:05.418003    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz" podUID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 10 04:00:06.421131 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:06.421096    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 04:00:06.421602 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:06.421469    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 04:00:06.422048 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:06.422027    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 04:00:09.988664 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:09.988626    2561 scope.go:117] "RemoveContainer" containerID="164b66385160128b5b703e2d3739ae851e1f813111e1ffa1e837e08cef6535fa"
Jul 10 04:00:10.386721 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:10.386688    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47" event={"ID":"b303a4c5-c22d-4751-9713-4e297bce4b66","Type":"ContainerStarted","Data":"dcde6413b163d88ea3b4b280ee3bc717ca05678e2150370c43ce73976d3ccd9d"}
Jul 10 04:00:11.394545 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:11.394508    2561 generic.go:356] "Generic (PLEG): container finished" podID="b303a4c5-c22d-4751-9713-4e297bce4b66" containerID="dcde6413b163d88ea3b4b280ee3bc717ca05678e2150370c43ce73976d3ccd9d" exitCode=0
Jul 10 04:00:11.395001 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:11.394580    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47" event={"ID":"b303a4c5-c22d-4751-9713-4e297bce4b66","Type":"ContainerDied","Data":"dcde6413b163d88ea3b4b280ee3bc717ca05678e2150370c43ce73976d3ccd9d"}
Jul 10 04:00:11.395001 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:11.394642    2561 scope.go:117] "RemoveContainer" containerID="164b66385160128b5b703e2d3739ae851e1f813111e1ffa1e837e08cef6535fa"
Jul 10 04:00:11.410602 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:11.410578    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"]
Jul 10 04:00:12.538992 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:12.538969    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"
Jul 10 04:00:12.599471 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:12.599444    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b303a4c5-c22d-4751-9713-4e297bce4b66-maas-api-service-ca\") pod \"b303a4c5-c22d-4751-9713-4e297bce4b66\" (UID: \"b303a4c5-c22d-4751-9713-4e297bce4b66\") "
Jul 10 04:00:12.599821 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:12.599801    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b303a4c5-c22d-4751-9713-4e297bce4b66-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b303a4c5-c22d-4751-9713-4e297bce4b66" (UID: "b303a4c5-c22d-4751-9713-4e297bce4b66"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:00:12.700857 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:12.700767    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b303a4c5-c22d-4751-9713-4e297bce4b66-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:00:13.408721 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:13.408693    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"
Jul 10 04:00:13.408891 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:13.408687    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47" event={"ID":"b303a4c5-c22d-4751-9713-4e297bce4b66","Type":"ContainerDied","Data":"cf206a4e67554c64d6c35616d82a6ab3a589b66c3eac34c7bae076a476aba244"}
Jul 10 04:00:13.408891 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:13.408819    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf206a4e67554c64d6c35616d82a6ab3a589b66c3eac34c7bae076a476aba244"
Jul 10 04:00:13.433330 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:13.433295    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"]
Jul 10 04:00:13.435505 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:13.435479    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-xqk47"]
Jul 10 04:00:14.993739 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:14.993706    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b303a4c5-c22d-4751-9713-4e297bce4b66" path="/var/lib/kubelet/pods/b303a4c5-c22d-4751-9713-4e297bce4b66/volumes"
Jul 10 04:00:44.412407 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412318    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6"]
Jul 10 04:00:44.412922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412763    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06f0e816-96a6-4cd0-a90b-75cb9fcb90df" containerName="cleanup"
Jul 10 04:00:44.412922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412778    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="06f0e816-96a6-4cd0-a90b-75cb9fcb90df" containerName="cleanup"
Jul 10 04:00:44.412922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412793    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1c68fd1-41e4-460b-9b40-1c1d66d08bce" containerName="cleanup"
Jul 10 04:00:44.412922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412799    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1c68fd1-41e4-460b-9b40-1c1d66d08bce" containerName="cleanup"
Jul 10 04:00:44.412922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412815    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b303a4c5-c22d-4751-9713-4e297bce4b66" containerName="revoke-keys"
Jul 10 04:00:44.412922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412821    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b303a4c5-c22d-4751-9713-4e297bce4b66" containerName="revoke-keys"
Jul 10 04:00:44.412922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412829    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b303a4c5-c22d-4751-9713-4e297bce4b66" containerName="revoke-keys"
Jul 10 04:00:44.412922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412835    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b303a4c5-c22d-4751-9713-4e297bce4b66" containerName="revoke-keys"
Jul 10 04:00:44.412922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412911    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b303a4c5-c22d-4751-9713-4e297bce4b66" containerName="revoke-keys"
Jul 10 04:00:44.412922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412919    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b303a4c5-c22d-4751-9713-4e297bce4b66" containerName="revoke-keys"
Jul 10 04:00:44.412922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412927    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="06f0e816-96a6-4cd0-a90b-75cb9fcb90df" containerName="cleanup"
Jul 10 04:00:44.413336 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.412935    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1c68fd1-41e4-460b-9b40-1c1d66d08bce" containerName="cleanup"
Jul 10 04:00:44.416056 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.416039    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6"
Jul 10 04:00:44.418445 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.418422    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:00:44.427749 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.427721    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6"]
Jul 10 04:00:44.490115 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.490063    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-11393920-g45c6\" (UID: \"bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6"
Jul 10 04:00:44.590859 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.590818    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-11393920-g45c6\" (UID: \"bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6"
Jul 10 04:00:44.591417 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.591398    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-11393920-g45c6\" (UID: \"bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6"
Jul 10 04:00:44.728414 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.728330    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6"
Jul 10 04:00:44.865752 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:00:44.865709    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5b1e0e_5e7b_4ac6_bd0b_20e18f2e0d9a.slice/crio-b1c02d9739dba4c27e65b8a874d9cda5562a6a2fd4551416fe673606222e4c11 WatchSource:0}: Error finding container b1c02d9739dba4c27e65b8a874d9cda5562a6a2fd4551416fe673606222e4c11: Status 404 returned error can't find the container with id b1c02d9739dba4c27e65b8a874d9cda5562a6a2fd4551416fe673606222e4c11
Jul 10 04:00:44.865884 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:44.865806    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6"]
Jul 10 04:00:45.580810 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:45.580771    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6" event={"ID":"bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a","Type":"ContainerStarted","Data":"13583d57148aea2f52c0a6a20b080f291407bd90d1cbd5d76cbd52cbd1e0ec61"}
Jul 10 04:00:45.581258 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:45.580834    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6" event={"ID":"bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a","Type":"ContainerStarted","Data":"b1c02d9739dba4c27e65b8a874d9cda5562a6a2fd4551416fe673606222e4c11"}
Jul 10 04:00:45.596401 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:45.596354    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6" podStartSLOduration=1.5963386339999999 podStartE2EDuration="1.596338634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:00:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:00:45.594912463 +0000 UTC m=+1707.085908356" watchObservedRunningTime="2026-07-10 04:00:45.596338634 +0000 UTC m=+1707.087334506"
Jul 10 04:00:46.587402 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:46.587365    2561 generic.go:356] "Generic (PLEG): container finished" podID="bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a" containerID="13583d57148aea2f52c0a6a20b080f291407bd90d1cbd5d76cbd52cbd1e0ec61" exitCode=0
Jul 10 04:00:46.587872 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:46.587407    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6" event={"ID":"bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a","Type":"ContainerDied","Data":"13583d57148aea2f52c0a6a20b080f291407bd90d1cbd5d76cbd52cbd1e0ec61"}
Jul 10 04:00:47.733592 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:47.733565    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6"
Jul 10 04:00:47.817192 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:47.817151    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a-maas-api-service-ca\") pod \"bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a\" (UID: \"bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a\") "
Jul 10 04:00:47.817530 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:47.817503    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a" (UID: "bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:00:47.918133 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:47.918048    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:00:48.600074 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:48.600039    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6" event={"ID":"bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a","Type":"ContainerDied","Data":"b1c02d9739dba4c27e65b8a874d9cda5562a6a2fd4551416fe673606222e4c11"}
Jul 10 04:00:48.600074 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:48.600071    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-11393920-g45c6"
Jul 10 04:00:48.600280 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:48.600093    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1c02d9739dba4c27e65b8a874d9cda5562a6a2fd4551416fe673606222e4c11"
Jul 10 04:00:54.764523 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:54.764488    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"]
Jul 10 04:00:54.764971 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:54.764750    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8" podUID="00186fae-0c69-47e4-bde5-913f436ff2d7" containerName="maas-api" containerID="cri-o://c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755" gracePeriod=30
Jul 10 04:00:54.788793 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:54.788763    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g"]
Jul 10 04:00:54.792539 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:54.792497    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:00:54.793801 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:54.793778    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-11393920-29727600-p7t4g"]
Jul 10 04:00:54.994994 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:54.994964    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06f0e816-96a6-4cd0-a90b-75cb9fcb90df" path="/var/lib/kubelet/pods/06f0e816-96a6-4cd0-a90b-75cb9fcb90df/volumes"
Jul 10 04:00:55.019206 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.019134    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 04:00:55.077225 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.077191    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00186fae-0c69-47e4-bde5-913f436ff2d7-openshift-service-ca\") pod \"00186fae-0c69-47e4-bde5-913f436ff2d7\" (UID: \"00186fae-0c69-47e4-bde5-913f436ff2d7\") "
Jul 10 04:00:55.077433 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.077255    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvsz7\" (UniqueName: \"kubernetes.io/projected/00186fae-0c69-47e4-bde5-913f436ff2d7-kube-api-access-qvsz7\") pod \"00186fae-0c69-47e4-bde5-913f436ff2d7\" (UID: \"00186fae-0c69-47e4-bde5-913f436ff2d7\") "
Jul 10 04:00:55.077433 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.077286    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00186fae-0c69-47e4-bde5-913f436ff2d7-maas-api-tls\") pod \"00186fae-0c69-47e4-bde5-913f436ff2d7\" (UID: \"00186fae-0c69-47e4-bde5-913f436ff2d7\") "
Jul 10 04:00:55.077629 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.077601    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00186fae-0c69-47e4-bde5-913f436ff2d7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "00186fae-0c69-47e4-bde5-913f436ff2d7" (UID: "00186fae-0c69-47e4-bde5-913f436ff2d7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:00:55.079526 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.079500    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00186fae-0c69-47e4-bde5-913f436ff2d7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "00186fae-0c69-47e4-bde5-913f436ff2d7" (UID: "00186fae-0c69-47e4-bde5-913f436ff2d7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:00:55.079637 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.079549    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00186fae-0c69-47e4-bde5-913f436ff2d7-kube-api-access-qvsz7" (OuterVolumeSpecName: "kube-api-access-qvsz7") pod "00186fae-0c69-47e4-bde5-913f436ff2d7" (UID: "00186fae-0c69-47e4-bde5-913f436ff2d7"). InnerVolumeSpecName "kube-api-access-qvsz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:00:55.178950 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.178908    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvsz7\" (UniqueName: \"kubernetes.io/projected/00186fae-0c69-47e4-bde5-913f436ff2d7-kube-api-access-qvsz7\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:00:55.178950 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.178941    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00186fae-0c69-47e4-bde5-913f436ff2d7-maas-api-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:00:55.178950 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.178955    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00186fae-0c69-47e4-bde5-913f436ff2d7-openshift-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:00:55.649256 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.649226    2561 generic.go:356] "Generic (PLEG): container finished" podID="00186fae-0c69-47e4-bde5-913f436ff2d7" containerID="c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755" exitCode=0
Jul 10 04:00:55.649444 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.649273    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"
Jul 10 04:00:55.649444 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.649316    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8" event={"ID":"00186fae-0c69-47e4-bde5-913f436ff2d7","Type":"ContainerDied","Data":"c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755"}
Jul 10 04:00:55.649444 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.649370    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8" event={"ID":"00186fae-0c69-47e4-bde5-913f436ff2d7","Type":"ContainerDied","Data":"60af13e6ac6743184467a615ea9097f0fbcbb2f9598546d4dc2a24209e4fc067"}
Jul 10 04:00:55.649444 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.649393    2561 scope.go:117] "RemoveContainer" containerID="c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755"
Jul 10 04:00:55.659290 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.659270    2561 scope.go:117] "RemoveContainer" containerID="c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755"
Jul 10 04:00:55.659520 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:00:55.659502    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755\": container with ID starting with c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755 not found: ID does not exist" containerID="c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755"
Jul 10 04:00:55.659579 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.659529    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755"} err="failed to get container status \"c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755\": rpc error: code = NotFound desc = could not find container \"c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755\": container with ID starting with c3711afdf49a004003b6d1ba4733d28279de20c3cd542ada6b083177c0f6f755 not found: ID does not exist"
Jul 10 04:00:55.670568 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.670509    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"]
Jul 10 04:00:55.674951 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:55.674924    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-11393920-768777dd74-7p6b8"]
Jul 10 04:00:56.994188 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:56.994157    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00186fae-0c69-47e4-bde5-913f436ff2d7" path="/var/lib/kubelet/pods/00186fae-0c69-47e4-bde5-913f436ff2d7/volumes"
Jul 10 04:00:59.485166 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:00:59.485129    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:05.088670 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:05.088613    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:05.157719 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:05.157685    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:09.654165 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:09.654127    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"]
Jul 10 04:01:09.654640 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:09.654392    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz" podUID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerName="istio-proxy" containerID="cri-o://550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9" gracePeriod=30
Jul 10 04:01:09.674553 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:09.674518    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:10.010063 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:10.010016    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:11.121124 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.121090    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:11.208137 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.208082    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:11.210134 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.210104    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"]
Jul 10 04:01:11.210514 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.210498    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a" containerName="revoke-keys"
Jul 10 04:01:11.210580 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.210516    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a" containerName="revoke-keys"
Jul 10 04:01:11.210580 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.210530    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b303a4c5-c22d-4751-9713-4e297bce4b66" containerName="revoke-keys"
Jul 10 04:01:11.210580 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.210536    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b303a4c5-c22d-4751-9713-4e297bce4b66" containerName="revoke-keys"
Jul 10 04:01:11.210580 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.210552    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00186fae-0c69-47e4-bde5-913f436ff2d7" containerName="maas-api"
Jul 10 04:01:11.210580 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.210557    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="00186fae-0c69-47e4-bde5-913f436ff2d7" containerName="maas-api"
Jul 10 04:01:11.210792 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.210616    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="00186fae-0c69-47e4-bde5-913f436ff2d7" containerName="maas-api"
Jul 10 04:01:11.210792 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.210630    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b303a4c5-c22d-4751-9713-4e297bce4b66" containerName="revoke-keys"
Jul 10 04:01:11.210792 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.210636    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a" containerName="revoke-keys"
Jul 10 04:01:11.215230 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.215208    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:11.217531 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.217505    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8d8a2105-serving-cert\""
Jul 10 04:01:11.217682 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.217536    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 04:01:11.218373 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.218356    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 04:01:11.218464 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.218385    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6g9gw\""
Jul 10 04:01:11.223724 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.223586    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"]
Jul 10 04:01:11.277246 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.277208    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:11.323440 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.323407    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/857b28d7-3f91-494b-8532-4bae24d93786-maas-api-tls\") pod \"maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:11.323596 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.323451    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqm26\" (UniqueName: \"kubernetes.io/projected/857b28d7-3f91-494b-8532-4bae24d93786-kube-api-access-vqm26\") pod \"maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:11.323596 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.323550    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/857b28d7-3f91-494b-8532-4bae24d93786-openshift-service-ca\") pod \"maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:11.425056 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.424971    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/857b28d7-3f91-494b-8532-4bae24d93786-openshift-service-ca\") pod \"maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:11.425201 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.425093    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/857b28d7-3f91-494b-8532-4bae24d93786-maas-api-tls\") pod \"maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:11.425201 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.425138    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqm26\" (UniqueName: \"kubernetes.io/projected/857b28d7-3f91-494b-8532-4bae24d93786-kube-api-access-vqm26\") pod \"maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:11.425278 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:01:11.425252    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-serving-cert: secret "maas-api-e2e-ait-8d8a2105-serving-cert" not found
Jul 10 04:01:11.425340 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:01:11.425328    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857b28d7-3f91-494b-8532-4bae24d93786-maas-api-tls podName:857b28d7-3f91-494b-8532-4bae24d93786 nodeName:}" failed. No retries permitted until 2026-07-10 04:01:11.925307788 +0000 UTC m=+1733.416303643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/857b28d7-3f91-494b-8532-4bae24d93786-maas-api-tls") pod "maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45" (UID: "857b28d7-3f91-494b-8532-4bae24d93786") : secret "maas-api-e2e-ait-8d8a2105-serving-cert" not found
Jul 10 04:01:11.425721 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.425703    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/857b28d7-3f91-494b-8532-4bae24d93786-openshift-service-ca\") pod \"maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:11.435309 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.435286    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqm26\" (UniqueName: \"kubernetes.io/projected/857b28d7-3f91-494b-8532-4bae24d93786-kube-api-access-vqm26\") pod \"maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:11.598716 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.598680    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:11.766288 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.766193    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:11.882319 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.882287    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:11.929516 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.929476    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/857b28d7-3f91-494b-8532-4bae24d93786-maas-api-tls\") pod \"maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:11.931799 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:11.931780    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/857b28d7-3f91-494b-8532-4bae24d93786-maas-api-tls\") pod \"maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:12.127939 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:12.127908    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:12.262985 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:12.262961    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"]
Jul 10 04:01:12.265299 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:01:12.265270    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod857b28d7_3f91_494b_8532_4bae24d93786.slice/crio-1908a11276705ba16dbd11e20b4be93fb69d3af54b3289bacd81bf4d72228ca9 WatchSource:0}: Error finding container 1908a11276705ba16dbd11e20b4be93fb69d3af54b3289bacd81bf4d72228ca9: Status 404 returned error can't find the container with id 1908a11276705ba16dbd11e20b4be93fb69d3af54b3289bacd81bf4d72228ca9
Jul 10 04:01:12.750449 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:12.750413    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45" event={"ID":"857b28d7-3f91-494b-8532-4bae24d93786","Type":"ContainerStarted","Data":"7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45"}
Jul 10 04:01:12.750585 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:12.750491    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45" event={"ID":"857b28d7-3f91-494b-8532-4bae24d93786","Type":"ContainerStarted","Data":"1908a11276705ba16dbd11e20b4be93fb69d3af54b3289bacd81bf4d72228ca9"}
Jul 10 04:01:12.750585 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:12.750554    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:12.773730 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:12.773678    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45" podStartSLOduration=1.382411638 podStartE2EDuration="1.773639781s" totalImagesPullingTime="391.228143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:01:12.771859468 +0000 UTC m=+1734.262855355" watchObservedRunningTime="2026-07-10 04:01:12.773639781 +0000 UTC m=+1734.264635714"
Jul 10 04:01:14.902577 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.902550    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 04:01:14.958017 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.957986    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-envoy\") pod \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") "
Jul 10 04:01:14.958017 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958031    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-workload-socket\") pod \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") "
Jul 10 04:01:14.958297 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958066    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq9dg\" (UniqueName: \"kubernetes.io/projected/1fe65fe7-1b0d-462f-9d79-6822befae7cc-kube-api-access-vq9dg\") pod \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") "
Jul 10 04:01:14.958297 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958084    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-data\") pod \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") "
Jul 10 04:01:14.958297 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958104    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-credential-socket\") pod \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") "
Jul 10 04:01:14.958297 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958126    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-workload-certs\") pod \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") "
Jul 10 04:01:14.958297 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958152    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-podinfo\") pod \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") "
Jul 10 04:01:14.958297 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958211    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istiod-ca-cert\") pod \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") "
Jul 10 04:01:14.958297 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958250    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-token\") pod \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\" (UID: \"1fe65fe7-1b0d-462f-9d79-6822befae7cc\") "
Jul 10 04:01:14.958689 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958458    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1fe65fe7-1b0d-462f-9d79-6822befae7cc" (UID: "1fe65fe7-1b0d-462f-9d79-6822befae7cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:01:14.958689 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958540    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-credential-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:01:14.958810 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958708    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1fe65fe7-1b0d-462f-9d79-6822befae7cc" (UID: "1fe65fe7-1b0d-462f-9d79-6822befae7cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:01:14.958868 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958812    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1fe65fe7-1b0d-462f-9d79-6822befae7cc" (UID: "1fe65fe7-1b0d-462f-9d79-6822befae7cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:01:14.958943 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.958900    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "1fe65fe7-1b0d-462f-9d79-6822befae7cc" (UID: "1fe65fe7-1b0d-462f-9d79-6822befae7cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:01:14.959201 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.959174    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1fe65fe7-1b0d-462f-9d79-6822befae7cc" (UID: "1fe65fe7-1b0d-462f-9d79-6822befae7cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:01:14.960974 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.960905    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1fe65fe7-1b0d-462f-9d79-6822befae7cc" (UID: "1fe65fe7-1b0d-462f-9d79-6822befae7cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:01:14.961086 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.961027    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1fe65fe7-1b0d-462f-9d79-6822befae7cc" (UID: "1fe65fe7-1b0d-462f-9d79-6822befae7cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 04:01:14.961086 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.961031    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "1fe65fe7-1b0d-462f-9d79-6822befae7cc" (UID: "1fe65fe7-1b0d-462f-9d79-6822befae7cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:01:14.961511 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:14.961486    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fe65fe7-1b0d-462f-9d79-6822befae7cc-kube-api-access-vq9dg" (OuterVolumeSpecName: "kube-api-access-vq9dg") pod "1fe65fe7-1b0d-462f-9d79-6822befae7cc" (UID: "1fe65fe7-1b0d-462f-9d79-6822befae7cc"). InnerVolumeSpecName "kube-api-access-vq9dg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:01:15.059323 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.059287    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-envoy\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:01:15.059323 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.059319    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-workload-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:01:15.059323 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.059328    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq9dg\" (UniqueName: \"kubernetes.io/projected/1fe65fe7-1b0d-462f-9d79-6822befae7cc-kube-api-access-vq9dg\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:01:15.059323 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.059338    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-data\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:01:15.059598 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.059349    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fe65fe7-1b0d-462f-9d79-6822befae7cc-workload-certs\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:01:15.059598 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.059361    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-podinfo\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:01:15.059598 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.059372    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istiod-ca-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:01:15.059598 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.059386    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fe65fe7-1b0d-462f-9d79-6822befae7cc-istio-token\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:01:15.769039 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.769007    2561 generic.go:356] "Generic (PLEG): container finished" podID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerID="550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9" exitCode=0
Jul 10 04:01:15.769447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.769052    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"
Jul 10 04:01:15.769447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.769066    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz" event={"ID":"1fe65fe7-1b0d-462f-9d79-6822befae7cc","Type":"ContainerDied","Data":"550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9"}
Jul 10 04:01:15.769447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.769108    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz" event={"ID":"1fe65fe7-1b0d-462f-9d79-6822befae7cc","Type":"ContainerDied","Data":"a75aa5da30f8783765c860cb1b09cc8aaccc935d0ddaed9b673d6bcc8ab12550"}
Jul 10 04:01:15.769447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.769129    2561 scope.go:117] "RemoveContainer" containerID="550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9"
Jul 10 04:01:15.778638 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.778438    2561 scope.go:117] "RemoveContainer" containerID="fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6"
Jul 10 04:01:15.788000 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.787979    2561 scope.go:117] "RemoveContainer" containerID="550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9"
Jul 10 04:01:15.788137 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.788120    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"]
Jul 10 04:01:15.788259 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:01:15.788242    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9\": container with ID starting with 550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9 not found: ID does not exist" containerID="550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9"
Jul 10 04:01:15.788331 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.788271    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9"} err="failed to get container status \"550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9\": rpc error: code = NotFound desc = could not find container \"550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9\": container with ID starting with 550837f6103a1ac20b1a17753210e64de81c64809493f4297fef21aa9aa1fee9 not found: ID does not exist"
Jul 10 04:01:15.788331 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.788297    2561 scope.go:117] "RemoveContainer" containerID="fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6"
Jul 10 04:01:15.788546 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:01:15.788530    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6\": container with ID starting with fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6 not found: ID does not exist" containerID="fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6"
Jul 10 04:01:15.788600 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.788555    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6"} err="failed to get container status \"fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6\": rpc error: code = NotFound desc = could not find container \"fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6\": container with ID starting with fea662634ace455c792bbf7a17845cf7568e217f2ecdd2de8a8d60e80048aea6 not found: ID does not exist"
Jul 10 04:01:15.790508 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.790488    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-11393920-openshift-default-6576f7c87f-xj7bz"]
Jul 10 04:01:15.974077 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:15.974037    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:16.076336 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.076295    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:16.282615 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.282579    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"]
Jul 10 04:01:16.282982 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.282969    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerName="istio-proxy"
Jul 10 04:01:16.283026 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.282983    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerName="istio-proxy"
Jul 10 04:01:16.283066 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.283046    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerName="istio-proxy"
Jul 10 04:01:16.283066 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.283056    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerName="istio-proxy"
Jul 10 04:01:16.287184 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.287166    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"
Jul 10 04:01:16.289824 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.289796    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:01:16.293498 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.293465    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"]
Jul 10 04:01:16.371235 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.371149    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7d5493-c788-4c63-a31a-01da6de4da98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd\" (UID: \"6d7d5493-c788-4c63-a31a-01da6de4da98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"
Jul 10 04:01:16.472447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.472405    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7d5493-c788-4c63-a31a-01da6de4da98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd\" (UID: \"6d7d5493-c788-4c63-a31a-01da6de4da98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"
Jul 10 04:01:16.473049 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.473030    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7d5493-c788-4c63-a31a-01da6de4da98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd\" (UID: \"6d7d5493-c788-4c63-a31a-01da6de4da98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"
Jul 10 04:01:16.599421 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.599384    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"
Jul 10 04:01:16.738098 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.738069    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"]
Jul 10 04:01:16.740422 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:01:16.740393    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d7d5493_c788_4c63_a31a_01da6de4da98.slice/crio-104a86032555843d9df1113bae4f553100a33ab41cff9cf42ea7a3a569299b5c WatchSource:0}: Error finding container 104a86032555843d9df1113bae4f553100a33ab41cff9cf42ea7a3a569299b5c: Status 404 returned error can't find the container with id 104a86032555843d9df1113bae4f553100a33ab41cff9cf42ea7a3a569299b5c
Jul 10 04:01:16.777172 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.777142    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd" event={"ID":"6d7d5493-c788-4c63-a31a-01da6de4da98","Type":"ContainerStarted","Data":"104a86032555843d9df1113bae4f553100a33ab41cff9cf42ea7a3a569299b5c"}
Jul 10 04:01:16.994789 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:16.994712    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" path="/var/lib/kubelet/pods/1fe65fe7-1b0d-462f-9d79-6822befae7cc/volumes"
Jul 10 04:01:17.784477 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:17.784438    2561 generic.go:356] "Generic (PLEG): container finished" podID="6d7d5493-c788-4c63-a31a-01da6de4da98" containerID="9a0a98ddb7c6c5e6e095eb19d68974aea55bac050b52deff8ec12aa3ab63bbad" exitCode=7
Jul 10 04:01:17.784671 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:17.784527    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd" event={"ID":"6d7d5493-c788-4c63-a31a-01da6de4da98","Type":"ContainerDied","Data":"9a0a98ddb7c6c5e6e095eb19d68974aea55bac050b52deff8ec12aa3ab63bbad"}
Jul 10 04:01:17.784743 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:17.784729    2561 scope.go:117] "RemoveContainer" containerID="9a0a98ddb7c6c5e6e095eb19d68974aea55bac050b52deff8ec12aa3ab63bbad"
Jul 10 04:01:18.762307 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:18.762230    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:01:18.792741 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:18.792712    2561 generic.go:356] "Generic (PLEG): container finished" podID="6d7d5493-c788-4c63-a31a-01da6de4da98" containerID="e8901d75aa925de8339e2f87c06387cfd5228d9696510b33a1e8af5dd324f1eb" exitCode=7
Jul 10 04:01:18.792918 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:18.792785    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd" event={"ID":"6d7d5493-c788-4c63-a31a-01da6de4da98","Type":"ContainerDied","Data":"e8901d75aa925de8339e2f87c06387cfd5228d9696510b33a1e8af5dd324f1eb"}
Jul 10 04:01:18.792918 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:18.792828    2561 scope.go:117] "RemoveContainer" containerID="9a0a98ddb7c6c5e6e095eb19d68974aea55bac050b52deff8ec12aa3ab63bbad"
Jul 10 04:01:18.793041 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:18.793002    2561 scope.go:117] "RemoveContainer" containerID="e8901d75aa925de8339e2f87c06387cfd5228d9696510b33a1e8af5dd324f1eb"
Jul 10 04:01:18.793296 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:01:18.793264    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd_odh-ai-gateway-infra(6d7d5493-c788-4c63-a31a-01da6de4da98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd" podUID="6d7d5493-c788-4c63-a31a-01da6de4da98"
Jul 10 04:01:19.381511 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:19.381474    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:01:19.799410 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:19.799382    2561 scope.go:117] "RemoveContainer" containerID="e8901d75aa925de8339e2f87c06387cfd5228d9696510b33a1e8af5dd324f1eb"
Jul 10 04:01:19.799798 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:01:19.799578    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd_odh-ai-gateway-infra(6d7d5493-c788-4c63-a31a-01da6de4da98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd" podUID="6d7d5493-c788-4c63-a31a-01da6de4da98"
Jul 10 04:01:31.988747 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:31.988718    2561 scope.go:117] "RemoveContainer" containerID="e8901d75aa925de8339e2f87c06387cfd5228d9696510b33a1e8af5dd324f1eb"
Jul 10 04:01:32.877450 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:32.877412    2561 generic.go:356] "Generic (PLEG): container finished" podID="6d7d5493-c788-4c63-a31a-01da6de4da98" containerID="1c0fba62fc9450fed4af1dfabbe5a3ccaf93db5088ad1d976a75dd362a70ee52" exitCode=0
Jul 10 04:01:32.877596 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:32.877471    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd" event={"ID":"6d7d5493-c788-4c63-a31a-01da6de4da98","Type":"ContainerDied","Data":"1c0fba62fc9450fed4af1dfabbe5a3ccaf93db5088ad1d976a75dd362a70ee52"}
Jul 10 04:01:32.877596 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:32.877536    2561 scope.go:117] "RemoveContainer" containerID="e8901d75aa925de8339e2f87c06387cfd5228d9696510b33a1e8af5dd324f1eb"
Jul 10 04:01:33.901601 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:33.901565    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"]
Jul 10 04:01:34.024344 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:34.024315    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"
Jul 10 04:01:34.132737 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:34.132697    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7d5493-c788-4c63-a31a-01da6de4da98-maas-api-service-ca\") pod \"6d7d5493-c788-4c63-a31a-01da6de4da98\" (UID: \"6d7d5493-c788-4c63-a31a-01da6de4da98\") "
Jul 10 04:01:34.133086 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:34.133060    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d7d5493-c788-4c63-a31a-01da6de4da98-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6d7d5493-c788-4c63-a31a-01da6de4da98" (UID: "6d7d5493-c788-4c63-a31a-01da6de4da98"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:01:34.234291 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:34.234202    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7d5493-c788-4c63-a31a-01da6de4da98-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:01:34.892222 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:34.892188    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"
Jul 10 04:01:34.892222 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:34.892207    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd" event={"ID":"6d7d5493-c788-4c63-a31a-01da6de4da98","Type":"ContainerDied","Data":"104a86032555843d9df1113bae4f553100a33ab41cff9cf42ea7a3a569299b5c"}
Jul 10 04:01:34.892448 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:34.892249    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="104a86032555843d9df1113bae4f553100a33ab41cff9cf42ea7a3a569299b5c"
Jul 10 04:01:34.919951 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:34.919918    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"]
Jul 10 04:01:34.922777 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:34.922754    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-w87kd"]
Jul 10 04:01:34.994779 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:34.994749    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d7d5493-c788-4c63-a31a-01da6de4da98" path="/var/lib/kubelet/pods/6d7d5493-c788-4c63-a31a-01da6de4da98/volumes"
Jul 10 04:01:36.032023 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:01:36.031988    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:06.322506 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.322470    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs"]
Jul 10 04:02:06.322949 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.322862    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d7d5493-c788-4c63-a31a-01da6de4da98" containerName="revoke-keys"
Jul 10 04:02:06.322949 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.322875    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d7d5493-c788-4c63-a31a-01da6de4da98" containerName="revoke-keys"
Jul 10 04:02:06.322949 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.322885    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerName="istio-proxy"
Jul 10 04:02:06.322949 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.322891    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fe65fe7-1b0d-462f-9d79-6822befae7cc" containerName="istio-proxy"
Jul 10 04:02:06.322949 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.322901    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d7d5493-c788-4c63-a31a-01da6de4da98" containerName="revoke-keys"
Jul 10 04:02:06.322949 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.322907    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d7d5493-c788-4c63-a31a-01da6de4da98" containerName="revoke-keys"
Jul 10 04:02:06.323142 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.322965    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d7d5493-c788-4c63-a31a-01da6de4da98" containerName="revoke-keys"
Jul 10 04:02:06.323142 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.322973    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d7d5493-c788-4c63-a31a-01da6de4da98" containerName="revoke-keys"
Jul 10 04:02:06.326146 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.326129    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs"
Jul 10 04:02:06.328564 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.328540    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:02:06.332330 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.332304    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs"]
Jul 10 04:02:06.415732 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.415631    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89d9a6a4-c002-465d-a30e-e6909f228104-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs\" (UID: \"89d9a6a4-c002-465d-a30e-e6909f228104\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs"
Jul 10 04:02:06.516973 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.516925    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89d9a6a4-c002-465d-a30e-e6909f228104-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs\" (UID: \"89d9a6a4-c002-465d-a30e-e6909f228104\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs"
Jul 10 04:02:06.517585 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.517566    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89d9a6a4-c002-465d-a30e-e6909f228104-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs\" (UID: \"89d9a6a4-c002-465d-a30e-e6909f228104\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs"
Jul 10 04:02:06.638909 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.638820    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs"
Jul 10 04:02:06.979790 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:06.979764    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs"]
Jul 10 04:02:06.981484 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:02:06.981455    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89d9a6a4_c002_465d_a30e_e6909f228104.slice/crio-6f706cb2628a3ac57080293deb7197ce14ae017fe8452c194201fb400630c97a WatchSource:0}: Error finding container 6f706cb2628a3ac57080293deb7197ce14ae017fe8452c194201fb400630c97a: Status 404 returned error can't find the container with id 6f706cb2628a3ac57080293deb7197ce14ae017fe8452c194201fb400630c97a
Jul 10 04:02:07.066597 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:07.066565    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs" event={"ID":"89d9a6a4-c002-465d-a30e-e6909f228104","Type":"ContainerStarted","Data":"6f706cb2628a3ac57080293deb7197ce14ae017fe8452c194201fb400630c97a"}
Jul 10 04:02:08.073442 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:08.073407    2561 generic.go:356] "Generic (PLEG): container finished" podID="89d9a6a4-c002-465d-a30e-e6909f228104" containerID="43783aad87147b94240c628fcfe8c51bd84adf84577e845ebba839e8bef1baed" exitCode=0
Jul 10 04:02:08.073842 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:08.073489    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs" event={"ID":"89d9a6a4-c002-465d-a30e-e6909f228104","Type":"ContainerDied","Data":"43783aad87147b94240c628fcfe8c51bd84adf84577e845ebba839e8bef1baed"}
Jul 10 04:02:09.211908 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:09.211886    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs"
Jul 10 04:02:09.338988 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:09.338902    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89d9a6a4-c002-465d-a30e-e6909f228104-maas-api-service-ca\") pod \"89d9a6a4-c002-465d-a30e-e6909f228104\" (UID: \"89d9a6a4-c002-465d-a30e-e6909f228104\") "
Jul 10 04:02:09.339267 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:09.339244    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89d9a6a4-c002-465d-a30e-e6909f228104-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "89d9a6a4-c002-465d-a30e-e6909f228104" (UID: "89d9a6a4-c002-465d-a30e-e6909f228104"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:02:09.439642 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:09.439588    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89d9a6a4-c002-465d-a30e-e6909f228104-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:02:10.085318 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:10.085287    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs"
Jul 10 04:02:10.085497 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:10.085281    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs" event={"ID":"89d9a6a4-c002-465d-a30e-e6909f228104","Type":"ContainerDied","Data":"6f706cb2628a3ac57080293deb7197ce14ae017fe8452c194201fb400630c97a"}
Jul 10 04:02:10.085497 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:10.085421    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f706cb2628a3ac57080293deb7197ce14ae017fe8452c194201fb400630c97a"
Jul 10 04:02:16.690899 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:16.690868    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"]
Jul 10 04:02:16.691353 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:16.691082    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45" podUID="857b28d7-3f91-494b-8532-4bae24d93786" containerName="maas-api" containerID="cri-o://7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45" gracePeriod=30
Jul 10 04:02:16.718423 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:16.718386    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:16.937287 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:16.937260    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:02:17.016005 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.015911    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/857b28d7-3f91-494b-8532-4bae24d93786-openshift-service-ca\") pod \"857b28d7-3f91-494b-8532-4bae24d93786\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") "
Jul 10 04:02:17.016005 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.015994    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqm26\" (UniqueName: \"kubernetes.io/projected/857b28d7-3f91-494b-8532-4bae24d93786-kube-api-access-vqm26\") pod \"857b28d7-3f91-494b-8532-4bae24d93786\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") "
Jul 10 04:02:17.016253 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.016062    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/857b28d7-3f91-494b-8532-4bae24d93786-maas-api-tls\") pod \"857b28d7-3f91-494b-8532-4bae24d93786\" (UID: \"857b28d7-3f91-494b-8532-4bae24d93786\") "
Jul 10 04:02:17.016314 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.016294    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/857b28d7-3f91-494b-8532-4bae24d93786-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "857b28d7-3f91-494b-8532-4bae24d93786" (UID: "857b28d7-3f91-494b-8532-4bae24d93786"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:02:17.018125 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.018103    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/857b28d7-3f91-494b-8532-4bae24d93786-kube-api-access-vqm26" (OuterVolumeSpecName: "kube-api-access-vqm26") pod "857b28d7-3f91-494b-8532-4bae24d93786" (UID: "857b28d7-3f91-494b-8532-4bae24d93786"). InnerVolumeSpecName "kube-api-access-vqm26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:02:17.018306 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.018284    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/857b28d7-3f91-494b-8532-4bae24d93786-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "857b28d7-3f91-494b-8532-4bae24d93786" (UID: "857b28d7-3f91-494b-8532-4bae24d93786"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:02:17.117608 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.117567    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqm26\" (UniqueName: \"kubernetes.io/projected/857b28d7-3f91-494b-8532-4bae24d93786-kube-api-access-vqm26\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:02:17.117608 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.117602    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/857b28d7-3f91-494b-8532-4bae24d93786-maas-api-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:02:17.117608 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.117614    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/857b28d7-3f91-494b-8532-4bae24d93786-openshift-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:02:17.129813 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.129784    2561 generic.go:356] "Generic (PLEG): container finished" podID="857b28d7-3f91-494b-8532-4bae24d93786" containerID="7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45" exitCode=0
Jul 10 04:02:17.129944 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.129831    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"
Jul 10 04:02:17.129944 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.129864    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45" event={"ID":"857b28d7-3f91-494b-8532-4bae24d93786","Type":"ContainerDied","Data":"7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45"}
Jul 10 04:02:17.129944 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.129924    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45" event={"ID":"857b28d7-3f91-494b-8532-4bae24d93786","Type":"ContainerDied","Data":"1908a11276705ba16dbd11e20b4be93fb69d3af54b3289bacd81bf4d72228ca9"}
Jul 10 04:02:17.130096 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.129953    2561 scope.go:117] "RemoveContainer" containerID="7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45"
Jul 10 04:02:17.153833 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.153802    2561 scope.go:117] "RemoveContainer" containerID="7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45"
Jul 10 04:02:17.154202 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:02:17.154171    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45\": container with ID starting with 7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45 not found: ID does not exist" containerID="7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45"
Jul 10 04:02:17.154310 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.154213    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45"} err="failed to get container status \"7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45\": rpc error: code = NotFound desc = could not find container \"7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45\": container with ID starting with 7ac5d0b39b992becb67c219a2b61a854250010f4d4bf0e6e8736095c96927b45 not found: ID does not exist"
Jul 10 04:02:17.165136 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.165052    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"]
Jul 10 04:02:17.168175 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:17.168146    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8d8a2105-5dfd75c497-wgt45"]
Jul 10 04:02:18.994354 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:18.994324    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="857b28d7-3f91-494b-8532-4bae24d93786" path="/var/lib/kubelet/pods/857b28d7-3f91-494b-8532-4bae24d93786/volumes"
Jul 10 04:02:19.139815 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:19.139790    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 04:02:19.146719 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:19.146690    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 04:02:21.399829 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:21.399763    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:27.198914 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:27.198877    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:27.302141 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:27.302107    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:32.444534 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.444497    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:32.731352 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.731266    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"]
Jul 10 04:02:32.731631 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.731619    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="857b28d7-3f91-494b-8532-4bae24d93786" containerName="maas-api"
Jul 10 04:02:32.731747 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.731632    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="857b28d7-3f91-494b-8532-4bae24d93786" containerName="maas-api"
Jul 10 04:02:32.731747 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.731642    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d7d5493-c788-4c63-a31a-01da6de4da98" containerName="revoke-keys"
Jul 10 04:02:32.731747 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.731709    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d7d5493-c788-4c63-a31a-01da6de4da98" containerName="revoke-keys"
Jul 10 04:02:32.731747 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.731718    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89d9a6a4-c002-465d-a30e-e6909f228104" containerName="revoke-keys"
Jul 10 04:02:32.731747 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.731723    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="89d9a6a4-c002-465d-a30e-e6909f228104" containerName="revoke-keys"
Jul 10 04:02:32.732010 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.731807    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="857b28d7-3f91-494b-8532-4bae24d93786" containerName="maas-api"
Jul 10 04:02:32.732010 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.731813    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="89d9a6a4-c002-465d-a30e-e6909f228104" containerName="revoke-keys"
Jul 10 04:02:32.732010 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.731820    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d7d5493-c788-4c63-a31a-01da6de4da98" containerName="revoke-keys"
Jul 10 04:02:32.734640 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.734619    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.737097 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.737075    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 04:02:32.737214 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.737106    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-6b62d0e3-openshift-default-dockercfg-ck2b2\""
Jul 10 04:02:32.748349 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.748319    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"]
Jul 10 04:02:32.752351 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.752325    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-workload-socket\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.752351 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.752353    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-credential-socket\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.752536 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.752374    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-data\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.752536 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.752446    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istiod-ca-cert\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.752536 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.752495    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-workload-certs\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.752536 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.752523    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8vll\" (UniqueName: \"kubernetes.io/projected/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-kube-api-access-r8vll\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.752800 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.752562    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-envoy\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.752800 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.752582    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-podinfo\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.752800 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.752602    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-token\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.768953 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.768929    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:32.853240 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.853199    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istiod-ca-cert\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.853447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.853246    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-workload-certs\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.853447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.853273    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8vll\" (UniqueName: \"kubernetes.io/projected/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-kube-api-access-r8vll\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.853447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.853298    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-envoy\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.853447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.853318    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-podinfo\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.853447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.853340    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-token\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.853447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.853425    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-workload-socket\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.853811 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.853450    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-credential-socket\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.853811 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.853487    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-data\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.853811 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.853728    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-workload-certs\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.853974 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.853918    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-data\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.854026 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.853992    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-workload-socket\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.854117 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.854094    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-credential-socket\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.854159 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.854130    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istiod-ca-cert\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.855732 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.855711    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-envoy\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.855884 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.855864    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-podinfo\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.861728 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.861701    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8vll\" (UniqueName: \"kubernetes.io/projected/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-kube-api-access-r8vll\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:32.861913 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:32.861895    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-token\") pod \"e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") " pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:33.046348 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.046238    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:33.187031 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.187002    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"]
Jul 10 04:02:33.188174 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:02:33.188147    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be013d2_d136_41d3_8d1d_2dbbb5f11b0f.slice/crio-89bdedbfeb0a893ed63d307f8fbec4696974c26439068df6dfa1408ba27337fb WatchSource:0}: Error finding container 89bdedbfeb0a893ed63d307f8fbec4696974c26439068df6dfa1408ba27337fb: Status 404 returned error can't find the container with id 89bdedbfeb0a893ed63d307f8fbec4696974c26439068df6dfa1408ba27337fb
Jul 10 04:02:33.190174 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.190135    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:02:33.190298 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.190198    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:02:33.190298 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.190236    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:02:33.225049 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.225016    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" event={"ID":"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f","Type":"ContainerStarted","Data":"89bdedbfeb0a893ed63d307f8fbec4696974c26439068df6dfa1408ba27337fb"}
Jul 10 04:02:33.876549 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.876517    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:33.942152 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.942119    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"]
Jul 10 04:02:33.945878 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.945855    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:33.948930 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.948899    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-6b62d0e3-serving-cert\""
Jul 10 04:02:33.948930 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.948918    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 04:02:33.949430 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.949410    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6g9gw\""
Jul 10 04:02:33.949545 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.949499    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 04:02:33.956796 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.956774    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"]
Jul 10 04:02:33.962563 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.962537    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:33.963836 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.963806    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa97fc12-5812-477d-86cd-f94b56690d4a-openshift-service-ca\") pod \"maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:33.963967 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.963923    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa97fc12-5812-477d-86cd-f94b56690d4a-maas-api-tls\") pod \"maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:33.964070 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:33.964028    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8h6b\" (UniqueName: \"kubernetes.io/projected/aa97fc12-5812-477d-86cd-f94b56690d4a-kube-api-access-j8h6b\") pod \"maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:34.049233 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.049181    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:34.065033 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.064995    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa97fc12-5812-477d-86cd-f94b56690d4a-openshift-service-ca\") pod \"maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:34.065228 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.065040    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa97fc12-5812-477d-86cd-f94b56690d4a-maas-api-tls\") pod \"maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:34.065228 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.065106    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8h6b\" (UniqueName: \"kubernetes.io/projected/aa97fc12-5812-477d-86cd-f94b56690d4a-kube-api-access-j8h6b\") pod \"maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:34.065356 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:02:34.065232    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-serving-cert: secret "maas-api-e2e-derive-6b62d0e3-serving-cert" not found
Jul 10 04:02:34.065356 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:02:34.065304    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa97fc12-5812-477d-86cd-f94b56690d4a-maas-api-tls podName:aa97fc12-5812-477d-86cd-f94b56690d4a nodeName:}" failed. No retries permitted until 2026-07-10 04:02:34.565282631 +0000 UTC m=+1816.056278487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aa97fc12-5812-477d-86cd-f94b56690d4a-maas-api-tls") pod "maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6" (UID: "aa97fc12-5812-477d-86cd-f94b56690d4a") : secret "maas-api-e2e-derive-6b62d0e3-serving-cert" not found
Jul 10 04:02:34.065720 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.065697    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa97fc12-5812-477d-86cd-f94b56690d4a-openshift-service-ca\") pod \"maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:34.083121 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.083088    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8h6b\" (UniqueName: \"kubernetes.io/projected/aa97fc12-5812-477d-86cd-f94b56690d4a-kube-api-access-j8h6b\") pod \"maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:34.234718 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.234594    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" event={"ID":"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f","Type":"ContainerStarted","Data":"47248c2401322b376e7f749972701ab9d15da12c0eb3d58962a48c6aa195528f"}
Jul 10 04:02:34.278261 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.278199    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" podStartSLOduration=2.278181925 podStartE2EDuration="2.278181925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:02:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:02:34.275961762 +0000 UTC m=+1815.766957658" watchObservedRunningTime="2026-07-10 04:02:34.278181925 +0000 UTC m=+1815.769177797"
Jul 10 04:02:34.468091 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.468052    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:34.569438 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.569353    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa97fc12-5812-477d-86cd-f94b56690d4a-maas-api-tls\") pod \"maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:34.571759 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.571737    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa97fc12-5812-477d-86cd-f94b56690d4a-maas-api-tls\") pod \"maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:34.752154 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.752116    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:34.853690 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.853625    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:34.859494 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:34.859461    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:35.003283 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:35.003233    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"]
Jul 10 04:02:35.004592 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:02:35.004562    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa97fc12_5812_477d_86cd_f94b56690d4a.slice/crio-c671926ebc2f8cb3398b3fafa094af1b6ec41d678fd15995bf2761ba522c64a5 WatchSource:0}: Error finding container c671926ebc2f8cb3398b3fafa094af1b6ec41d678fd15995bf2761ba522c64a5: Status 404 returned error can't find the container with id c671926ebc2f8cb3398b3fafa094af1b6ec41d678fd15995bf2761ba522c64a5
Jul 10 04:02:35.046817 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:35.046774    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:35.053035 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:35.053004    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:35.241642 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:35.241563    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6" event={"ID":"aa97fc12-5812-477d-86cd-f94b56690d4a","Type":"ContainerStarted","Data":"c671926ebc2f8cb3398b3fafa094af1b6ec41d678fd15995bf2761ba522c64a5"}
Jul 10 04:02:35.241934 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:35.241917    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:35.242820 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:35.242800    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:36.249024 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:36.248989    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6" event={"ID":"aa97fc12-5812-477d-86cd-f94b56690d4a","Type":"ContainerStarted","Data":"0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793"}
Jul 10 04:02:36.249461 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:36.249205    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:36.266435 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:36.266385    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6" podStartSLOduration=2.777410439 podStartE2EDuration="3.266369665s" totalImagesPullingTime="488.959226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:02:36.263729702 +0000 UTC m=+1817.754725601" watchObservedRunningTime="2026-07-10 04:02:36.266369665 +0000 UTC m=+1817.757365537"
Jul 10 04:02:38.305419 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:38.305380    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"]
Jul 10 04:02:38.309023 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:38.309006    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"
Jul 10 04:02:38.311218 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:38.311195    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:02:38.315973 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:38.315950    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"]
Jul 10 04:02:38.405042 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:38.405004    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b27d43f-8d0c-4313-b10d-4700f37836b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877\" (UID: \"0b27d43f-8d0c-4313-b10d-4700f37836b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"
Jul 10 04:02:38.505921 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:38.505883    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b27d43f-8d0c-4313-b10d-4700f37836b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877\" (UID: \"0b27d43f-8d0c-4313-b10d-4700f37836b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"
Jul 10 04:02:38.506519 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:38.506499    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b27d43f-8d0c-4313-b10d-4700f37836b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877\" (UID: \"0b27d43f-8d0c-4313-b10d-4700f37836b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"
Jul 10 04:02:38.622024 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:38.621990    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"
Jul 10 04:02:38.741160 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:38.741128    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:38.842184 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:38.842147    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:38.972372 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:38.972324    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"]
Jul 10 04:02:38.974662 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:02:38.974611    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b27d43f_8d0c_4313_b10d_4700f37836b9.slice/crio-850a447c0747067d7c84e98eb8e505f789977f1e36a7dec38195a375b5debaf2 WatchSource:0}: Error finding container 850a447c0747067d7c84e98eb8e505f789977f1e36a7dec38195a375b5debaf2: Status 404 returned error can't find the container with id 850a447c0747067d7c84e98eb8e505f789977f1e36a7dec38195a375b5debaf2
Jul 10 04:02:39.266830 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:39.266739    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877" event={"ID":"0b27d43f-8d0c-4313-b10d-4700f37836b9","Type":"ContainerStarted","Data":"850a447c0747067d7c84e98eb8e505f789977f1e36a7dec38195a375b5debaf2"}
Jul 10 04:02:40.273812 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:40.273777    2561 generic.go:356] "Generic (PLEG): container finished" podID="0b27d43f-8d0c-4313-b10d-4700f37836b9" containerID="0fb8fae14a9c25574ef10266150a06ad812c94ba585b6ccd1a2a31ed1600e4f5" exitCode=7
Jul 10 04:02:40.274207 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:40.273867    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877" event={"ID":"0b27d43f-8d0c-4313-b10d-4700f37836b9","Type":"ContainerDied","Data":"0fb8fae14a9c25574ef10266150a06ad812c94ba585b6ccd1a2a31ed1600e4f5"}
Jul 10 04:02:40.274207 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:40.274071    2561 scope.go:117] "RemoveContainer" containerID="0fb8fae14a9c25574ef10266150a06ad812c94ba585b6ccd1a2a31ed1600e4f5"
Jul 10 04:02:41.281437 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:41.281349    2561 generic.go:356] "Generic (PLEG): container finished" podID="0b27d43f-8d0c-4313-b10d-4700f37836b9" containerID="6b0b13baa8445aa65f92dc397137910a9e33c07810849f7425e32c26eb095c84" exitCode=7
Jul 10 04:02:41.281437 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:41.281404    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877" event={"ID":"0b27d43f-8d0c-4313-b10d-4700f37836b9","Type":"ContainerDied","Data":"6b0b13baa8445aa65f92dc397137910a9e33c07810849f7425e32c26eb095c84"}
Jul 10 04:02:41.281926 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:41.281452    2561 scope.go:117] "RemoveContainer" containerID="0fb8fae14a9c25574ef10266150a06ad812c94ba585b6ccd1a2a31ed1600e4f5"
Jul 10 04:02:41.281926 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:41.281623    2561 scope.go:117] "RemoveContainer" containerID="6b0b13baa8445aa65f92dc397137910a9e33c07810849f7425e32c26eb095c84"
Jul 10 04:02:41.281926 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:02:41.281886    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877_odh-ai-gateway-infra(0b27d43f-8d0c-4313-b10d-4700f37836b9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877" podUID="0b27d43f-8d0c-4313-b10d-4700f37836b9"
Jul 10 04:02:42.261055 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:42.261024    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:02:42.289960 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:42.289935    2561 scope.go:117] "RemoveContainer" containerID="6b0b13baa8445aa65f92dc397137910a9e33c07810849f7425e32c26eb095c84"
Jul 10 04:02:42.290358 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:02:42.290137    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877_odh-ai-gateway-infra(0b27d43f-8d0c-4313-b10d-4700f37836b9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877" podUID="0b27d43f-8d0c-4313-b10d-4700f37836b9"
Jul 10 04:02:42.384443 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:42.384411    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:50.243604 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:50.243566    2561 patch_prober.go:28] interesting pod/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 10 04:02:50.244019 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:50.243625    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 10 04:02:50.346150 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:50.346115    2561 generic.go:356] "Generic (PLEG): container finished" podID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerID="47248c2401322b376e7f749972701ab9d15da12c0eb3d58962a48c6aa195528f" exitCode=0
Jul 10 04:02:50.346321 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:50.346188    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" event={"ID":"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f","Type":"ContainerDied","Data":"47248c2401322b376e7f749972701ab9d15da12c0eb3d58962a48c6aa195528f"}
Jul 10 04:02:50.346701 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:50.346683    2561 scope.go:117] "RemoveContainer" containerID="47248c2401322b376e7f749972701ab9d15da12c0eb3d58962a48c6aa195528f"
Jul 10 04:02:50.347819 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:50.347792    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:02:50.347888 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:50.347857    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:02:50.347922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:50.347889    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:02:51.353858 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:51.353816    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" event={"ID":"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f","Type":"ContainerStarted","Data":"4d2e85370418af3569c73df80b2ea16e74231edb7ac32f5b1faddc93b1f4e079"}
Jul 10 04:02:52.047033 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:52.046997    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:52.047250 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:52.047175    2561 patch_prober.go:28] interesting pod/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 10 04:02:52.047250 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:52.047229    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 10 04:02:53.047113 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:53.047066    2561 patch_prober.go:28] interesting pod/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 10 04:02:53.047591 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:53.047141    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 10 04:02:54.046698 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:54.046639    2561 patch_prober.go:28] interesting pod/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 10 04:02:54.046885 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:54.046728    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 10 04:02:55.047559 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:55.047518    2561 patch_prober.go:28] interesting pod/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 10 04:02:55.047990 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:55.047577    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 10 04:02:55.988780 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:55.988751    2561 scope.go:117] "RemoveContainer" containerID="6b0b13baa8445aa65f92dc397137910a9e33c07810849f7425e32c26eb095c84"
Jul 10 04:02:56.050875 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:56.050837    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:56.051320 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:56.051236    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:56.052013 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:56.051991    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:02:56.387284 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:56.387250    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877" event={"ID":"0b27d43f-8d0c-4313-b10d-4700f37836b9","Type":"ContainerStarted","Data":"f27db778996a00edd8f19c5df451ba07033fcd51e5415f3473e82db397b6101f"}
Jul 10 04:02:56.403536 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:56.403490    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877" podStartSLOduration=18.40347534 podStartE2EDuration="18.40347534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:02:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:02:56.40076039 +0000 UTC m=+1837.891756274" watchObservedRunningTime="2026-07-10 04:02:56.40347534 +0000 UTC m=+1837.894471224"
Jul 10 04:02:57.053276 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:57.053194    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:57.151133 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:57.151084    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:02:57.394964 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:57.394928    2561 generic.go:356] "Generic (PLEG): container finished" podID="0b27d43f-8d0c-4313-b10d-4700f37836b9" containerID="f27db778996a00edd8f19c5df451ba07033fcd51e5415f3473e82db397b6101f" exitCode=0
Jul 10 04:02:57.395171 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:57.394971    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877" event={"ID":"0b27d43f-8d0c-4313-b10d-4700f37836b9","Type":"ContainerDied","Data":"f27db778996a00edd8f19c5df451ba07033fcd51e5415f3473e82db397b6101f"}
Jul 10 04:02:57.395171 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:57.395033    2561 scope.go:117] "RemoveContainer" containerID="6b0b13baa8445aa65f92dc397137910a9e33c07810849f7425e32c26eb095c84"
Jul 10 04:02:57.411124 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:57.411097    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"]
Jul 10 04:02:58.544435 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:58.544404    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"
Jul 10 04:02:58.688065 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:58.687975    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b27d43f-8d0c-4313-b10d-4700f37836b9-maas-api-service-ca\") pod \"0b27d43f-8d0c-4313-b10d-4700f37836b9\" (UID: \"0b27d43f-8d0c-4313-b10d-4700f37836b9\") "
Jul 10 04:02:58.688359 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:58.688336    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b27d43f-8d0c-4313-b10d-4700f37836b9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0b27d43f-8d0c-4313-b10d-4700f37836b9" (UID: "0b27d43f-8d0c-4313-b10d-4700f37836b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:02:58.789546 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:58.789502    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b27d43f-8d0c-4313-b10d-4700f37836b9-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:02:59.409959 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:59.409927    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"
Jul 10 04:02:59.410146 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:59.409927    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877" event={"ID":"0b27d43f-8d0c-4313-b10d-4700f37836b9","Type":"ContainerDied","Data":"850a447c0747067d7c84e98eb8e505f789977f1e36a7dec38195a375b5debaf2"}
Jul 10 04:02:59.410146 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:59.410052    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="850a447c0747067d7c84e98eb8e505f789977f1e36a7dec38195a375b5debaf2"
Jul 10 04:02:59.441016 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:59.440985    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"]
Jul 10 04:02:59.443248 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:02:59.443223    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-kt877"]
Jul 10 04:03:00.994749 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:00.994709    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b27d43f-8d0c-4313-b10d-4700f37836b9" path="/var/lib/kubelet/pods/0b27d43f-8d0c-4313-b10d-4700f37836b9/volumes"
Jul 10 04:03:09.060293 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:09.060252    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:03:09.157800 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:09.157762    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:03:24.860356 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:24.860319    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:03:24.961779 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:24.961744    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:03:38.351898 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.351855    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt"]
Jul 10 04:03:38.352382 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.352253    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b27d43f-8d0c-4313-b10d-4700f37836b9" containerName="revoke-keys"
Jul 10 04:03:38.352382 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.352265    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b27d43f-8d0c-4313-b10d-4700f37836b9" containerName="revoke-keys"
Jul 10 04:03:38.352382 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.352279    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b27d43f-8d0c-4313-b10d-4700f37836b9" containerName="revoke-keys"
Jul 10 04:03:38.352382 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.352285    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b27d43f-8d0c-4313-b10d-4700f37836b9" containerName="revoke-keys"
Jul 10 04:03:38.352382 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.352355    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b27d43f-8d0c-4313-b10d-4700f37836b9" containerName="revoke-keys"
Jul 10 04:03:38.352382 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.352366    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b27d43f-8d0c-4313-b10d-4700f37836b9" containerName="revoke-keys"
Jul 10 04:03:38.352382 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.352375    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b27d43f-8d0c-4313-b10d-4700f37836b9" containerName="revoke-keys"
Jul 10 04:03:38.355724 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.355702    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt"
Jul 10 04:03:38.357981 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.357958    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:03:38.363741 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.363390    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt"]
Jul 10 04:03:38.448357 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.448314    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50116307-d0ea-414c-80f0-dca5d287a0ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt\" (UID: \"50116307-d0ea-414c-80f0-dca5d287a0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt"
Jul 10 04:03:38.549311 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.549268    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50116307-d0ea-414c-80f0-dca5d287a0ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt\" (UID: \"50116307-d0ea-414c-80f0-dca5d287a0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt"
Jul 10 04:03:38.550036 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.550016    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50116307-d0ea-414c-80f0-dca5d287a0ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt\" (UID: \"50116307-d0ea-414c-80f0-dca5d287a0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt"
Jul 10 04:03:38.668902 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.668808    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt"
Jul 10 04:03:38.806810 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:38.806782    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt"]
Jul 10 04:03:38.808956 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:03:38.808928    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50116307_d0ea_414c_80f0_dca5d287a0ff.slice/crio-5641220b46c04873fa33c15760d7131aff3d92591ddd0067504d8b4886bbdfd6 WatchSource:0}: Error finding container 5641220b46c04873fa33c15760d7131aff3d92591ddd0067504d8b4886bbdfd6: Status 404 returned error can't find the container with id 5641220b46c04873fa33c15760d7131aff3d92591ddd0067504d8b4886bbdfd6
Jul 10 04:03:39.640958 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:39.640922    2561 generic.go:356] "Generic (PLEG): container finished" podID="50116307-d0ea-414c-80f0-dca5d287a0ff" containerID="e54d81e831214bf7b8a798e0f27421761da97e09bff12177aa782153079e6b00" exitCode=0
Jul 10 04:03:39.641431 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:39.641005    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt" event={"ID":"50116307-d0ea-414c-80f0-dca5d287a0ff","Type":"ContainerDied","Data":"e54d81e831214bf7b8a798e0f27421761da97e09bff12177aa782153079e6b00"}
Jul 10 04:03:39.641431 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:39.641059    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt" event={"ID":"50116307-d0ea-414c-80f0-dca5d287a0ff","Type":"ContainerStarted","Data":"5641220b46c04873fa33c15760d7131aff3d92591ddd0067504d8b4886bbdfd6"}
Jul 10 04:03:40.781206 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:40.781180    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt"
Jul 10 04:03:40.870658 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:40.870623    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50116307-d0ea-414c-80f0-dca5d287a0ff-maas-api-service-ca\") pod \"50116307-d0ea-414c-80f0-dca5d287a0ff\" (UID: \"50116307-d0ea-414c-80f0-dca5d287a0ff\") "
Jul 10 04:03:40.871029 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:40.871009    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50116307-d0ea-414c-80f0-dca5d287a0ff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50116307-d0ea-414c-80f0-dca5d287a0ff" (UID: "50116307-d0ea-414c-80f0-dca5d287a0ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:03:40.972158 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:40.972062    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50116307-d0ea-414c-80f0-dca5d287a0ff-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:03:41.655056 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:41.655027    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt"
Jul 10 04:03:41.655232 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:41.655067    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt" event={"ID":"50116307-d0ea-414c-80f0-dca5d287a0ff","Type":"ContainerDied","Data":"5641220b46c04873fa33c15760d7131aff3d92591ddd0067504d8b4886bbdfd6"}
Jul 10 04:03:41.655232 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:41.655114    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5641220b46c04873fa33c15760d7131aff3d92591ddd0067504d8b4886bbdfd6"
Jul 10 04:03:48.722311 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:48.722277    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"]
Jul 10 04:03:48.722741 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:48.722537    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6" podUID="aa97fc12-5812-477d-86cd-f94b56690d4a" containerName="maas-api" containerID="cri-o://0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793" gracePeriod=30
Jul 10 04:03:48.741267 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:48.741240    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:03:48.975515 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:48.975449    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:03:49.048222 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.048188    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa97fc12-5812-477d-86cd-f94b56690d4a-openshift-service-ca\") pod \"aa97fc12-5812-477d-86cd-f94b56690d4a\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") "
Jul 10 04:03:49.048400 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.048272    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa97fc12-5812-477d-86cd-f94b56690d4a-maas-api-tls\") pod \"aa97fc12-5812-477d-86cd-f94b56690d4a\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") "
Jul 10 04:03:49.048400 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.048314    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8h6b\" (UniqueName: \"kubernetes.io/projected/aa97fc12-5812-477d-86cd-f94b56690d4a-kube-api-access-j8h6b\") pod \"aa97fc12-5812-477d-86cd-f94b56690d4a\" (UID: \"aa97fc12-5812-477d-86cd-f94b56690d4a\") "
Jul 10 04:03:49.048677 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.048615    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa97fc12-5812-477d-86cd-f94b56690d4a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa97fc12-5812-477d-86cd-f94b56690d4a" (UID: "aa97fc12-5812-477d-86cd-f94b56690d4a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:03:49.048924 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.048901    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa97fc12-5812-477d-86cd-f94b56690d4a-openshift-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:03:49.050526 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.050498    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa97fc12-5812-477d-86cd-f94b56690d4a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa97fc12-5812-477d-86cd-f94b56690d4a" (UID: "aa97fc12-5812-477d-86cd-f94b56690d4a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:03:49.050640 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.050589    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa97fc12-5812-477d-86cd-f94b56690d4a-kube-api-access-j8h6b" (OuterVolumeSpecName: "kube-api-access-j8h6b") pod "aa97fc12-5812-477d-86cd-f94b56690d4a" (UID: "aa97fc12-5812-477d-86cd-f94b56690d4a"). InnerVolumeSpecName "kube-api-access-j8h6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:03:49.149667 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.149622    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa97fc12-5812-477d-86cd-f94b56690d4a-maas-api-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:03:49.149838 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.149693    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8h6b\" (UniqueName: \"kubernetes.io/projected/aa97fc12-5812-477d-86cd-f94b56690d4a-kube-api-access-j8h6b\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:03:49.702013 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.701976    2561 generic.go:356] "Generic (PLEG): container finished" podID="aa97fc12-5812-477d-86cd-f94b56690d4a" containerID="0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793" exitCode=0
Jul 10 04:03:49.702211 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.702041    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"
Jul 10 04:03:49.702211 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.702058    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6" event={"ID":"aa97fc12-5812-477d-86cd-f94b56690d4a","Type":"ContainerDied","Data":"0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793"}
Jul 10 04:03:49.702211 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.702116    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6" event={"ID":"aa97fc12-5812-477d-86cd-f94b56690d4a","Type":"ContainerDied","Data":"c671926ebc2f8cb3398b3fafa094af1b6ec41d678fd15995bf2761ba522c64a5"}
Jul 10 04:03:49.702211 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.702143    2561 scope.go:117] "RemoveContainer" containerID="0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793"
Jul 10 04:03:49.711840 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.711818    2561 scope.go:117] "RemoveContainer" containerID="0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793"
Jul 10 04:03:49.712110 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:03:49.712090    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793\": container with ID starting with 0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793 not found: ID does not exist" containerID="0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793"
Jul 10 04:03:49.712169 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.712122    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793"} err="failed to get container status \"0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793\": rpc error: code = NotFound desc = could not find container \"0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793\": container with ID starting with 0210763d40bbe4b1bc8c9499470b4042bb26f5939032335dfa70135d81485793 not found: ID does not exist"
Jul 10 04:03:49.723596 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.723574    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"]
Jul 10 04:03:49.729325 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:49.729301    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6b62d0e3-586bb88949-w8rb6"]
Jul 10 04:03:50.994577 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:50.994536    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa97fc12-5812-477d-86cd-f94b56690d4a" path="/var/lib/kubelet/pods/aa97fc12-5812-477d-86cd-f94b56690d4a/volumes"
Jul 10 04:03:53.462863 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:53.462835    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:03:59.152489 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:59.152452    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:03:59.210381 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:03:59.210345    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:03.609168 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:03.609132    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"]
Jul 10 04:04:03.609615 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:03.609362    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerName="istio-proxy" containerID="cri-o://4d2e85370418af3569c73df80b2ea16e74231edb7ac32f5b1faddc93b1f4e079" gracePeriod=30
Jul 10 04:04:03.630910 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:03.630881    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:04.254707 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:04.254672    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:04.354700 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:04.354619    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:04.460359 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:04.460325    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:06.000532 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:06.000497    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:08.816473 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:08.816441    2561 generic.go:356] "Generic (PLEG): container finished" podID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerID="4d2e85370418af3569c73df80b2ea16e74231edb7ac32f5b1faddc93b1f4e079" exitCode=0
Jul 10 04:04:08.816892 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:08.816512    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" event={"ID":"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f","Type":"ContainerDied","Data":"4d2e85370418af3569c73df80b2ea16e74231edb7ac32f5b1faddc93b1f4e079"}
Jul 10 04:04:08.816892 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:08.816578    2561 scope.go:117] "RemoveContainer" containerID="47248c2401322b376e7f749972701ab9d15da12c0eb3d58962a48c6aa195528f"
Jul 10 04:04:08.873878 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:08.873818    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:04:09.030270 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030214    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istiod-ca-cert\") pod \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") "
Jul 10 04:04:09.030469 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030328    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-workload-socket\") pod \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") "
Jul 10 04:04:09.030469 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030355    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-data\") pod \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") "
Jul 10 04:04:09.030469 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030385    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-envoy\") pod \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") "
Jul 10 04:04:09.030469 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030431    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-token\") pod \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") "
Jul 10 04:04:09.030469 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030458    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-credential-socket\") pod \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") "
Jul 10 04:04:09.030763 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030493    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-workload-certs\") pod \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") "
Jul 10 04:04:09.030763 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030552    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8vll\" (UniqueName: \"kubernetes.io/projected/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-kube-api-access-r8vll\") pod \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") "
Jul 10 04:04:09.030763 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030576    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-podinfo\") pod \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\" (UID: \"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f\") "
Jul 10 04:04:09.030763 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030688    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" (UID: "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:04:09.030763 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030750    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" (UID: "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:04:09.031427 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030780    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" (UID: "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:04:09.031427 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030819    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" (UID: "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:04:09.031427 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030922    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-credential-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:04:09.031427 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030941    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-workload-certs\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:04:09.031427 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030953    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istiod-ca-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:04:09.031427 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.030964    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-workload-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:04:09.031840 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.031812    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-data" (OuterVolumeSpecName: "istio-data") pod "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" (UID: "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:04:09.032918 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.032895    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" (UID: "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:04:09.033102 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.033077    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-token" (OuterVolumeSpecName: "istio-token") pod "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" (UID: "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:04:09.033194 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.033172    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-kube-api-access-r8vll" (OuterVolumeSpecName: "kube-api-access-r8vll") pod "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" (UID: "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f"). InnerVolumeSpecName "kube-api-access-r8vll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:04:09.033431 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.033410    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" (UID: "5be013d2-d136-41d3-8d1d-2dbbb5f11b0f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 04:04:09.131683 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.131554    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-envoy\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:04:09.131683 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.131592    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-token\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:04:09.131683 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.131605    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8vll\" (UniqueName: \"kubernetes.io/projected/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-kube-api-access-r8vll\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:04:09.131683 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.131618    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-podinfo\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:04:09.131683 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.131630    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f-istio-data\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:04:09.824335 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.824307    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"
Jul 10 04:04:09.824796 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.824314    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd" event={"ID":"5be013d2-d136-41d3-8d1d-2dbbb5f11b0f","Type":"ContainerDied","Data":"89bdedbfeb0a893ed63d307f8fbec4696974c26439068df6dfa1408ba27337fb"}
Jul 10 04:04:09.824796 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.824434    2561 scope.go:117] "RemoveContainer" containerID="4d2e85370418af3569c73df80b2ea16e74231edb7ac32f5b1faddc93b1f4e079"
Jul 10 04:04:09.845220 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.845185    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"]
Jul 10 04:04:09.850503 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.850476    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-6b62d0e3-openshift-default-f5587d5ff-v97hd"]
Jul 10 04:04:09.957447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:09.957412    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:10.995856 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:10.995823    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" path="/var/lib/kubelet/pods/5be013d2-d136-41d3-8d1d-2dbbb5f11b0f/volumes"
Jul 10 04:04:11.815117 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:11.815084    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:11.872959 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:11.872926    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:11.952835 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:11.952782    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:12.052507 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:12.052474    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:12.152380 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:12.152347    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:12.586899 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:12.586867    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:38.587141 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:38.587110    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:44.291273 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:44.291238    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:44.347076 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:44.347045    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:44.431570 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:44.431532    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:04:44.530858 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:04:44.530805    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:00.470424 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:00.470388    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:06.195208 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:06.195173    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:06.302940 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:06.302903    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:06.395361 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:06.395318    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:16.974675 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:16.974562    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:17.807894 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:17.807855    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:23.370360 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:23.370315    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:23.431224 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:23.431192    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:23.519306 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:23.519275    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:23.613851 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:23.613819    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:23.714310 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:23.714226    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:24.507044 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:24.507005    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:24.606415 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:24.606378    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:24.709629 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:24.709594    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:26.398531 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:26.398495    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:32.141709 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:32.141673    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:32.242247 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:32.242205    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:32.836326 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:32.836294    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:38.471241 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:38.471177    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:38.569981 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:38.569940    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:38.711539 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:38.711505    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:39.023765 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.023733    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"]
Jul 10 04:05:39.024089 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024077    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa97fc12-5812-477d-86cd-f94b56690d4a" containerName="maas-api"
Jul 10 04:05:39.024144 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024090    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa97fc12-5812-477d-86cd-f94b56690d4a" containerName="maas-api"
Jul 10 04:05:39.024144 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024113    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerName="istio-proxy"
Jul 10 04:05:39.024144 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024118    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerName="istio-proxy"
Jul 10 04:05:39.024144 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024127    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50116307-d0ea-414c-80f0-dca5d287a0ff" containerName="revoke-keys"
Jul 10 04:05:39.024144 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024135    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="50116307-d0ea-414c-80f0-dca5d287a0ff" containerName="revoke-keys"
Jul 10 04:05:39.024342 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024148    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b27d43f-8d0c-4313-b10d-4700f37836b9" containerName="revoke-keys"
Jul 10 04:05:39.024342 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024156    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b27d43f-8d0c-4313-b10d-4700f37836b9" containerName="revoke-keys"
Jul 10 04:05:39.024342 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024165    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerName="istio-proxy"
Jul 10 04:05:39.024342 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024172    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerName="istio-proxy"
Jul 10 04:05:39.024342 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024253    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerName="istio-proxy"
Jul 10 04:05:39.024342 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024269    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa97fc12-5812-477d-86cd-f94b56690d4a" containerName="maas-api"
Jul 10 04:05:39.024342 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024285    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5be013d2-d136-41d3-8d1d-2dbbb5f11b0f" containerName="istio-proxy"
Jul 10 04:05:39.024342 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.024297    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="50116307-d0ea-414c-80f0-dca5d287a0ff" containerName="revoke-keys"
Jul 10 04:05:39.027417 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.027400    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.029633 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.029604    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 04:05:39.029759 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.029685    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-aa8dc0f1-openshift-default-dockercfg-5wxzv\""
Jul 10 04:05:39.049169 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.049138    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:39.050739 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.050720    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"]
Jul 10 04:05:39.129049 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.129014    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-credential-socket\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.129244 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.129078    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68292182-87c2-4954-84ad-568d97b1b254-istiod-ca-cert\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.129244 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.129109    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fst9q\" (UniqueName: \"kubernetes.io/projected/68292182-87c2-4954-84ad-568d97b1b254-kube-api-access-fst9q\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.129244 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.129141    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-workload-certs\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.129433 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.129258    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68292182-87c2-4954-84ad-568d97b1b254-istio-podinfo\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.129433 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.129307    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-workload-socket\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.129433 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.129341    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68292182-87c2-4954-84ad-568d97b1b254-istio-token\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.129543 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.129464    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-istio-envoy\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.129584 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.129535    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-istio-data\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.230323 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.230284    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-istio-envoy\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.230323 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.230325    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-istio-data\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.230592 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.230348    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-credential-socket\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.230592 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.230368    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68292182-87c2-4954-84ad-568d97b1b254-istiod-ca-cert\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.230592 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.230387    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fst9q\" (UniqueName: \"kubernetes.io/projected/68292182-87c2-4954-84ad-568d97b1b254-kube-api-access-fst9q\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.230592 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.230509    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-workload-certs\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.230592 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.230568    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68292182-87c2-4954-84ad-568d97b1b254-istio-podinfo\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.230895 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.230602    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-workload-socket\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.230895 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.230678    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68292182-87c2-4954-84ad-568d97b1b254-istio-token\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.230895 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.230869    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-istio-data\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.231057 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.230899    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-credential-socket\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.231057 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.230974    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-workload-certs\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.231164 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.231108    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-workload-socket\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.231220 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.231177    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68292182-87c2-4954-84ad-568d97b1b254-istiod-ca-cert\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.232773 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.232748    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-istio-envoy\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.233108 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.233092    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68292182-87c2-4954-84ad-568d97b1b254-istio-podinfo\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.234281 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.234258    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:39.237303 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.237279    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68292182-87c2-4954-84ad-568d97b1b254-istio-token\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.237436 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.237318    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fst9q\" (UniqueName: \"kubernetes.io/projected/68292182-87c2-4954-84ad-568d97b1b254-kube-api-access-fst9q\") pod \"e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") " pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.337779 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.337751    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:39.480760 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.480733    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"]
Jul 10 04:05:39.481775 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:05:39.481750    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68292182_87c2_4954_84ad_568d97b1b254.slice/crio-a65fd6cdb63c2bf4c9226ee9dc4b30455f984f0da082fd1e8bc2dd205480357d WatchSource:0}: Error finding container a65fd6cdb63c2bf4c9226ee9dc4b30455f984f0da082fd1e8bc2dd205480357d: Status 404 returned error can't find the container with id a65fd6cdb63c2bf4c9226ee9dc4b30455f984f0da082fd1e8bc2dd205480357d
Jul 10 04:05:39.484162 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.484128    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:05:39.484248 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.484198    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:05:39.484248 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:39.484225    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:05:40.330825 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:40.330785    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" event={"ID":"68292182-87c2-4954-84ad-568d97b1b254","Type":"ContainerStarted","Data":"10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e"}
Jul 10 04:05:40.331016 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:40.330850    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" event={"ID":"68292182-87c2-4954-84ad-568d97b1b254","Type":"ContainerStarted","Data":"a65fd6cdb63c2bf4c9226ee9dc4b30455f984f0da082fd1e8bc2dd205480357d"}
Jul 10 04:05:40.338773 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:40.338748    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:40.343672 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:40.343631    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:40.355236 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:40.355192    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" podStartSLOduration=1.3551787800000001 podStartE2EDuration="1.35517878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:05:40.352204974 +0000 UTC m=+2001.843200849" watchObservedRunningTime="2026-07-10 04:05:40.35517878 +0000 UTC m=+2001.846174652"
Jul 10 04:05:41.336478 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:41.336443    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:41.337354 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:41.337333    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:42.316942 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.316849    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:42.380860 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.380827    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"]
Jul 10 04:05:42.384428 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.384404    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:42.385426 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.385397    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:42.386541 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.386519    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6g9gw\""
Jul 10 04:05:42.386693 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.386544    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-aa8dc0f1-serving-cert\""
Jul 10 04:05:42.386853 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.386835    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 04:05:42.387277 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.387262    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 04:05:42.396432 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.396407    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"]
Jul 10 04:05:42.450986 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.450950    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:42.462085 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.462054    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfk4q\" (UniqueName: \"kubernetes.io/projected/5a6ddd52-8bcc-429a-bf11-500d92627c78-kube-api-access-qfk4q\") pod \"maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:42.462273 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.462250    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a6ddd52-8bcc-429a-bf11-500d92627c78-maas-api-tls\") pod \"maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:42.462340 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.462299    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6ddd52-8bcc-429a-bf11-500d92627c78-openshift-service-ca\") pod \"maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:42.563082 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.563041    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfk4q\" (UniqueName: \"kubernetes.io/projected/5a6ddd52-8bcc-429a-bf11-500d92627c78-kube-api-access-qfk4q\") pod \"maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:42.563294 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.563125    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a6ddd52-8bcc-429a-bf11-500d92627c78-maas-api-tls\") pod \"maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:42.563294 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.563162    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6ddd52-8bcc-429a-bf11-500d92627c78-openshift-service-ca\") pod \"maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:42.563294 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:05:42.563259    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-serving-cert: secret "maas-api-e2e-mt-aa8dc0f1-serving-cert" not found
Jul 10 04:05:42.563459 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:05:42.563329    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a6ddd52-8bcc-429a-bf11-500d92627c78-maas-api-tls podName:5a6ddd52-8bcc-429a-bf11-500d92627c78 nodeName:}" failed. No retries permitted until 2026-07-10 04:05:43.063310576 +0000 UTC m=+2004.554306432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a6ddd52-8bcc-429a-bf11-500d92627c78-maas-api-tls") pod "maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg" (UID: "5a6ddd52-8bcc-429a-bf11-500d92627c78") : secret "maas-api-e2e-mt-aa8dc0f1-serving-cert" not found
Jul 10 04:05:42.563884 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.563860    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6ddd52-8bcc-429a-bf11-500d92627c78-openshift-service-ca\") pod \"maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:42.574365 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.574338    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfk4q\" (UniqueName: \"kubernetes.io/projected/5a6ddd52-8bcc-429a-bf11-500d92627c78-kube-api-access-qfk4q\") pod \"maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:42.815695 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.815634    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:42.963339 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:42.963207    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:43.064625 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:43.064591    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:43.068864 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:43.068844    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a6ddd52-8bcc-429a-bf11-500d92627c78-maas-api-tls\") pod \"maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:43.072520 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:43.072493    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a6ddd52-8bcc-429a-bf11-500d92627c78-maas-api-tls\") pod \"maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:43.297853 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:43.297769    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:43.435596 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:43.435569    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"]
Jul 10 04:05:43.440971 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:05:43.440934    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a6ddd52_8bcc_429a_bf11_500d92627c78.slice/crio-0be7769332fdbd8eae064f459a0d3bf341332150619a5ecae28b286b7b79f651 WatchSource:0}: Error finding container 0be7769332fdbd8eae064f459a0d3bf341332150619a5ecae28b286b7b79f651: Status 404 returned error can't find the container with id 0be7769332fdbd8eae064f459a0d3bf341332150619a5ecae28b286b7b79f651
Jul 10 04:05:43.442972 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:43.442943    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 04:05:44.356524 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:44.356481    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg" event={"ID":"5a6ddd52-8bcc-429a-bf11-500d92627c78","Type":"ContainerStarted","Data":"a03bac8eb3c505247e25049ccb354b5b8f4ab453352f6bf1a11affc6634944ba"}
Jul 10 04:05:44.356732 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:44.356582    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg" event={"ID":"5a6ddd52-8bcc-429a-bf11-500d92627c78","Type":"ContainerStarted","Data":"0be7769332fdbd8eae064f459a0d3bf341332150619a5ecae28b286b7b79f651"}
Jul 10 04:05:44.356732 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:44.356637    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:44.375784 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:44.375736    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg" podStartSLOduration=1.8906703980000001 podStartE2EDuration="2.375721393s" totalImagesPullingTime="485.050995ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:05:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:05:44.374054006 +0000 UTC m=+2005.865049880" watchObservedRunningTime="2026-07-10 04:05:44.375721393 +0000 UTC m=+2005.866717268"
Jul 10 04:05:46.841185 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:46.841149    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:47.540128 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:47.540090    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:47.641523 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:47.641488    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:48.021838 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:48.021804    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:50.368712 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:50.368684    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:05:53.571933 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:53.571895    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:53.672586 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:53.672528    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:05:53.936003 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:53.935917    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn"]
Jul 10 04:05:53.939556 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:53.939534    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn"
Jul 10 04:05:53.942027 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:53.941996    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:05:53.947332 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:53.947304    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn"]
Jul 10 04:05:53.971363 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:53.971325    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f57935-fd31-47bd-80a4-f2c0a21f98d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn\" (UID: \"a8f57935-fd31-47bd-80a4-f2c0a21f98d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn"
Jul 10 04:05:54.072205 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:54.072162    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f57935-fd31-47bd-80a4-f2c0a21f98d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn\" (UID: \"a8f57935-fd31-47bd-80a4-f2c0a21f98d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn"
Jul 10 04:05:54.072856 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:54.072837    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f57935-fd31-47bd-80a4-f2c0a21f98d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn\" (UID: \"a8f57935-fd31-47bd-80a4-f2c0a21f98d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn"
Jul 10 04:05:54.253347 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:54.253256    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn"
Jul 10 04:05:54.400053 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:54.399965    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn"]
Jul 10 04:05:54.402933 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:05:54.402891    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8f57935_fd31_47bd_80a4_f2c0a21f98d9.slice/crio-007ba4db38eac0c21f4c8c9168800016aa5cb053328c3071e4dd7e8d93f45b5d WatchSource:0}: Error finding container 007ba4db38eac0c21f4c8c9168800016aa5cb053328c3071e4dd7e8d93f45b5d: Status 404 returned error can't find the container with id 007ba4db38eac0c21f4c8c9168800016aa5cb053328c3071e4dd7e8d93f45b5d
Jul 10 04:05:54.424925 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:54.424896    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn" event={"ID":"a8f57935-fd31-47bd-80a4-f2c0a21f98d9","Type":"ContainerStarted","Data":"007ba4db38eac0c21f4c8c9168800016aa5cb053328c3071e4dd7e8d93f45b5d"}
Jul 10 04:05:55.432301 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:55.432211    2561 generic.go:356] "Generic (PLEG): container finished" podID="a8f57935-fd31-47bd-80a4-f2c0a21f98d9" containerID="7987bc27ff579cdf2eb112c1388b385b19208e694bdefb6fae8c2d6f26bd0b52" exitCode=0
Jul 10 04:05:55.432723 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:55.432295    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn" event={"ID":"a8f57935-fd31-47bd-80a4-f2c0a21f98d9","Type":"ContainerDied","Data":"7987bc27ff579cdf2eb112c1388b385b19208e694bdefb6fae8c2d6f26bd0b52"}
Jul 10 04:05:56.338605 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:56.338557    2561 patch_prober.go:28] interesting pod/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 10 04:05:56.338846 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:56.338623    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" podUID="68292182-87c2-4954-84ad-568d97b1b254" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 10 04:05:56.442395 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:56.442360    2561 generic.go:356] "Generic (PLEG): container finished" podID="68292182-87c2-4954-84ad-568d97b1b254" containerID="10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e" exitCode=0
Jul 10 04:05:56.442884 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:56.442432    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" event={"ID":"68292182-87c2-4954-84ad-568d97b1b254","Type":"ContainerDied","Data":"10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e"}
Jul 10 04:05:56.442884 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:56.442849    2561 scope.go:117] "RemoveContainer" containerID="10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e"
Jul 10 04:05:56.444057 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:56.444025    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:05:56.444158 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:56.444096    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:05:56.444158 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:56.444134    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:05:56.607488 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:56.607403    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn"
Jul 10 04:05:56.697194 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:56.697151    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f57935-fd31-47bd-80a4-f2c0a21f98d9-maas-api-service-ca\") pod \"a8f57935-fd31-47bd-80a4-f2c0a21f98d9\" (UID: \"a8f57935-fd31-47bd-80a4-f2c0a21f98d9\") "
Jul 10 04:05:56.697574 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:56.697550    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8f57935-fd31-47bd-80a4-f2c0a21f98d9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a8f57935-fd31-47bd-80a4-f2c0a21f98d9" (UID: "a8f57935-fd31-47bd-80a4-f2c0a21f98d9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:05:56.798763 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:56.798720    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f57935-fd31-47bd-80a4-f2c0a21f98d9-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:05:57.457238 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:57.456460    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" event={"ID":"68292182-87c2-4954-84ad-568d97b1b254","Type":"ContainerStarted","Data":"42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0"}
Jul 10 04:05:57.460687 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:57.459718    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn" event={"ID":"a8f57935-fd31-47bd-80a4-f2c0a21f98d9","Type":"ContainerDied","Data":"007ba4db38eac0c21f4c8c9168800016aa5cb053328c3071e4dd7e8d93f45b5d"}
Jul 10 04:05:57.460687 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:57.459775    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="007ba4db38eac0c21f4c8c9168800016aa5cb053328c3071e4dd7e8d93f45b5d"
Jul 10 04:05:57.460687 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:57.459839    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn"
Jul 10 04:05:58.338413 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:58.338305    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:05:58.338633 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:58.338609    2561 patch_prober.go:28] interesting pod/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 10 04:05:58.338757 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:58.338699    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" podUID="68292182-87c2-4954-84ad-568d97b1b254" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 10 04:05:59.338270 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:59.338232    2561 patch_prober.go:28] interesting pod/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 10 04:05:59.338689 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:05:59.338289    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" podUID="68292182-87c2-4954-84ad-568d97b1b254" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 10 04:06:00.338961 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:00.338923    2561 patch_prober.go:28] interesting pod/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 10 04:06:00.339373 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:00.338982    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" podUID="68292182-87c2-4954-84ad-568d97b1b254" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 10 04:06:01.338394 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:01.338349    2561 patch_prober.go:28] interesting pod/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 10 04:06:01.338582 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:01.338414    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" podUID="68292182-87c2-4954-84ad-568d97b1b254" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 10 04:06:02.342696 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:02.342661    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:06:02.343091 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:02.343074    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:06:02.343699 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:02.343677    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:06:04.286820 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.286780    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"]
Jul 10 04:06:04.287236 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.287056    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg" podUID="5a6ddd52-8bcc-429a-bf11-500d92627c78" containerName="maas-api" containerID="cri-o://a03bac8eb3c505247e25049ccb354b5b8f4ab453352f6bf1a11affc6634944ba" gracePeriod=30
Jul 10 04:06:04.368410 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.368378    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:04.505992 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.505963    2561 generic.go:356] "Generic (PLEG): container finished" podID="5a6ddd52-8bcc-429a-bf11-500d92627c78" containerID="a03bac8eb3c505247e25049ccb354b5b8f4ab453352f6bf1a11affc6634944ba" exitCode=0
Jul 10 04:06:04.506145 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.506007    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg" event={"ID":"5a6ddd52-8bcc-429a-bf11-500d92627c78","Type":"ContainerDied","Data":"a03bac8eb3c505247e25049ccb354b5b8f4ab453352f6bf1a11affc6634944ba"}
Jul 10 04:06:04.539473 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.539419    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:06:04.575818 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.575789    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfk4q\" (UniqueName: \"kubernetes.io/projected/5a6ddd52-8bcc-429a-bf11-500d92627c78-kube-api-access-qfk4q\") pod \"5a6ddd52-8bcc-429a-bf11-500d92627c78\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") "
Jul 10 04:06:04.575939 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.575873    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6ddd52-8bcc-429a-bf11-500d92627c78-openshift-service-ca\") pod \"5a6ddd52-8bcc-429a-bf11-500d92627c78\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") "
Jul 10 04:06:04.576076 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.576048    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a6ddd52-8bcc-429a-bf11-500d92627c78-maas-api-tls\") pod \"5a6ddd52-8bcc-429a-bf11-500d92627c78\" (UID: \"5a6ddd52-8bcc-429a-bf11-500d92627c78\") "
Jul 10 04:06:04.576244 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.576216    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a6ddd52-8bcc-429a-bf11-500d92627c78-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a6ddd52-8bcc-429a-bf11-500d92627c78" (UID: "5a6ddd52-8bcc-429a-bf11-500d92627c78"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:06:04.576386 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.576370    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6ddd52-8bcc-429a-bf11-500d92627c78-openshift-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:06:04.578071 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.578047    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a6ddd52-8bcc-429a-bf11-500d92627c78-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a6ddd52-8bcc-429a-bf11-500d92627c78" (UID: "5a6ddd52-8bcc-429a-bf11-500d92627c78"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:06:04.578173 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.578071    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a6ddd52-8bcc-429a-bf11-500d92627c78-kube-api-access-qfk4q" (OuterVolumeSpecName: "kube-api-access-qfk4q") pod "5a6ddd52-8bcc-429a-bf11-500d92627c78" (UID: "5a6ddd52-8bcc-429a-bf11-500d92627c78"). InnerVolumeSpecName "kube-api-access-qfk4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:06:04.676808 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.676776    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a6ddd52-8bcc-429a-bf11-500d92627c78-maas-api-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:06:04.676808 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:04.676803    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfk4q\" (UniqueName: \"kubernetes.io/projected/5a6ddd52-8bcc-429a-bf11-500d92627c78-kube-api-access-qfk4q\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:06:05.514369 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:05.514339    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"
Jul 10 04:06:05.514860 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:05.514376    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg" event={"ID":"5a6ddd52-8bcc-429a-bf11-500d92627c78","Type":"ContainerDied","Data":"0be7769332fdbd8eae064f459a0d3bf341332150619a5ecae28b286b7b79f651"}
Jul 10 04:06:05.514860 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:05.514433    2561 scope.go:117] "RemoveContainer" containerID="a03bac8eb3c505247e25049ccb354b5b8f4ab453352f6bf1a11affc6634944ba"
Jul 10 04:06:05.529686 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:05.529638    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"]
Jul 10 04:06:05.532859 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:05.532835    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aa8dc0f1-78f4d46487-nmzhg"]
Jul 10 04:06:06.994266 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:06.994213    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a6ddd52-8bcc-429a-bf11-500d92627c78" path="/var/lib/kubelet/pods/5a6ddd52-8bcc-429a-bf11-500d92627c78/volumes"
Jul 10 04:06:09.012484 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:09.012447    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:14.746971 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:14.746938    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:14.845385 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:14.845351    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:19.217090 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:19.217055    2561 scope.go:117] "RemoveContainer" containerID="8de3b925fd0bd2378efa12d41a04a7eeab7c74c985ed09d895a1d647087323d1"
Jul 10 04:06:19.225987 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:19.225965    2561 scope.go:117] "RemoveContainer" containerID="dcde6413b163d88ea3b4b280ee3bc717ca05678e2150370c43ce73976d3ccd9d"
Jul 10 04:06:19.739761 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:19.739723    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"]
Jul 10 04:06:19.740107 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:19.740059    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" podUID="68292182-87c2-4954-84ad-568d97b1b254" containerName="istio-proxy" containerID="cri-o://42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0" gracePeriod=30
Jul 10 04:06:19.758397 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:19.758369    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:20.002184 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:20.002086    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:20.119568 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:20.119538    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:20.219789 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:20.219762    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:20.346754 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:20.346727    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:25.004897 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.004869    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:06:25.163127 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.163088    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68292182-87c2-4954-84ad-568d97b1b254-istio-token\") pod \"68292182-87c2-4954-84ad-568d97b1b254\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") "
Jul 10 04:06:25.163335 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.163152    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-istio-data\") pod \"68292182-87c2-4954-84ad-568d97b1b254\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") "
Jul 10 04:06:25.163335 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.163183    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68292182-87c2-4954-84ad-568d97b1b254-istiod-ca-cert\") pod \"68292182-87c2-4954-84ad-568d97b1b254\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") "
Jul 10 04:06:25.163335 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.163219    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68292182-87c2-4954-84ad-568d97b1b254-istio-podinfo\") pod \"68292182-87c2-4954-84ad-568d97b1b254\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") "
Jul 10 04:06:25.163335 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.163276    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-istio-envoy\") pod \"68292182-87c2-4954-84ad-568d97b1b254\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") "
Jul 10 04:06:25.163565 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.163385    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fst9q\" (UniqueName: \"kubernetes.io/projected/68292182-87c2-4954-84ad-568d97b1b254-kube-api-access-fst9q\") pod \"68292182-87c2-4954-84ad-568d97b1b254\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") "
Jul 10 04:06:25.163565 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.163434    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-workload-socket\") pod \"68292182-87c2-4954-84ad-568d97b1b254\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") "
Jul 10 04:06:25.163565 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.163474    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-credential-socket\") pod \"68292182-87c2-4954-84ad-568d97b1b254\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") "
Jul 10 04:06:25.163565 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.163504    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-workload-certs\") pod \"68292182-87c2-4954-84ad-568d97b1b254\" (UID: \"68292182-87c2-4954-84ad-568d97b1b254\") "
Jul 10 04:06:25.163854 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.163632    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68292182-87c2-4954-84ad-568d97b1b254-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "68292182-87c2-4954-84ad-568d97b1b254" (UID: "68292182-87c2-4954-84ad-568d97b1b254"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:06:25.163934 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.163909    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "68292182-87c2-4954-84ad-568d97b1b254" (UID: "68292182-87c2-4954-84ad-568d97b1b254"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:06:25.164114 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.163975    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68292182-87c2-4954-84ad-568d97b1b254-istiod-ca-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:06:25.164224 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.164155    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "68292182-87c2-4954-84ad-568d97b1b254" (UID: "68292182-87c2-4954-84ad-568d97b1b254"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:06:25.164224 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.164163    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "68292182-87c2-4954-84ad-568d97b1b254" (UID: "68292182-87c2-4954-84ad-568d97b1b254"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:06:25.164365 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.164240    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-istio-data" (OuterVolumeSpecName: "istio-data") pod "68292182-87c2-4954-84ad-568d97b1b254" (UID: "68292182-87c2-4954-84ad-568d97b1b254"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:06:25.165495 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.165473    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68292182-87c2-4954-84ad-568d97b1b254-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "68292182-87c2-4954-84ad-568d97b1b254" (UID: "68292182-87c2-4954-84ad-568d97b1b254"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 04:06:25.165716 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.165697    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "68292182-87c2-4954-84ad-568d97b1b254" (UID: "68292182-87c2-4954-84ad-568d97b1b254"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:06:25.165910 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.165890    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68292182-87c2-4954-84ad-568d97b1b254-kube-api-access-fst9q" (OuterVolumeSpecName: "kube-api-access-fst9q") pod "68292182-87c2-4954-84ad-568d97b1b254" (UID: "68292182-87c2-4954-84ad-568d97b1b254"). InnerVolumeSpecName "kube-api-access-fst9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:06:25.165958 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.165913    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68292182-87c2-4954-84ad-568d97b1b254-istio-token" (OuterVolumeSpecName: "istio-token") pod "68292182-87c2-4954-84ad-568d97b1b254" (UID: "68292182-87c2-4954-84ad-568d97b1b254"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:06:25.265206 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.265172    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68292182-87c2-4954-84ad-568d97b1b254-istio-token\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:06:25.265206 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.265200    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-istio-data\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:06:25.265206 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.265209    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68292182-87c2-4954-84ad-568d97b1b254-istio-podinfo\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:06:25.265430 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.265218    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-istio-envoy\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:06:25.265430 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.265227    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fst9q\" (UniqueName: \"kubernetes.io/projected/68292182-87c2-4954-84ad-568d97b1b254-kube-api-access-fst9q\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:06:25.265430 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.265236    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-workload-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:06:25.265430 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.265245    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-credential-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:06:25.265430 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.265252    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68292182-87c2-4954-84ad-568d97b1b254-workload-certs\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:06:25.628481 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.628444    2561 generic.go:356] "Generic (PLEG): container finished" podID="68292182-87c2-4954-84ad-568d97b1b254" containerID="42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0" exitCode=0
Jul 10 04:06:25.628675 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.628496    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"
Jul 10 04:06:25.628675 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.628535    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" event={"ID":"68292182-87c2-4954-84ad-568d97b1b254","Type":"ContainerDied","Data":"42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0"}
Jul 10 04:06:25.628675 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.628596    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx" event={"ID":"68292182-87c2-4954-84ad-568d97b1b254","Type":"ContainerDied","Data":"a65fd6cdb63c2bf4c9226ee9dc4b30455f984f0da082fd1e8bc2dd205480357d"}
Jul 10 04:06:25.628675 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.628625    2561 scope.go:117] "RemoveContainer" containerID="42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0"
Jul 10 04:06:25.638688 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.638674    2561 scope.go:117] "RemoveContainer" containerID="10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e"
Jul 10 04:06:25.646998 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.646980    2561 scope.go:117] "RemoveContainer" containerID="42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0"
Jul 10 04:06:25.647218 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:06:25.647203    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0\": container with ID starting with 42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0 not found: ID does not exist" containerID="42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0"
Jul 10 04:06:25.647285 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.647224    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0"} err="failed to get container status \"42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0\": rpc error: code = NotFound desc = could not find container \"42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0\": container with ID starting with 42ae718b2bc5a0d7a03eb225175c9a33ad7654571e11721fe7809ed7dfa1d8a0 not found: ID does not exist"
Jul 10 04:06:25.647285 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.647240    2561 scope.go:117] "RemoveContainer" containerID="10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e"
Jul 10 04:06:25.647445 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:06:25.647427    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e\": container with ID starting with 10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e not found: ID does not exist" containerID="10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e"
Jul 10 04:06:25.647484 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.647448    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e"} err="failed to get container status \"10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e\": rpc error: code = NotFound desc = could not find container \"10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e\": container with ID starting with 10b91d67afbfa43d6c61839aee7fcfcf91cb03a482a44afb631cadba39e4446e not found: ID does not exist"
Jul 10 04:06:25.671396 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.671367    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"]
Jul 10 04:06:25.685237 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.685204    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-aa8dc0f1-openshift-default-79668c4df9-n7lfx"]
Jul 10 04:06:25.967296 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:25.967216    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:26.077010 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:26.076978    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:26.167413 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:26.167378    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:26.346114 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:26.346083    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:26.994280 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:26.994248    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68292182-87c2-4954-84ad-568d97b1b254" path="/var/lib/kubelet/pods/68292182-87c2-4954-84ad-568d97b1b254/volumes"
Jul 10 04:06:31.943358 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:31.943302    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:32.003044 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:32.003014    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:38.872792 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:38.872752    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:39.160734 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.160634    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"]
Jul 10 04:06:39.161206 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.161190    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a6ddd52-8bcc-429a-bf11-500d92627c78" containerName="maas-api"
Jul 10 04:06:39.161266 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.161209    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a6ddd52-8bcc-429a-bf11-500d92627c78" containerName="maas-api"
Jul 10 04:06:39.161266 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.161239    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68292182-87c2-4954-84ad-568d97b1b254" containerName="istio-proxy"
Jul 10 04:06:39.161266 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.161247    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="68292182-87c2-4954-84ad-568d97b1b254" containerName="istio-proxy"
Jul 10 04:06:39.161266 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.161258    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8f57935-fd31-47bd-80a4-f2c0a21f98d9" containerName="revoke-keys"
Jul 10 04:06:39.161399 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.161267    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8f57935-fd31-47bd-80a4-f2c0a21f98d9" containerName="revoke-keys"
Jul 10 04:06:39.161399 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.161278    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68292182-87c2-4954-84ad-568d97b1b254" containerName="istio-proxy"
Jul 10 04:06:39.161399 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.161287    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="68292182-87c2-4954-84ad-568d97b1b254" containerName="istio-proxy"
Jul 10 04:06:39.161399 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.161362    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="68292182-87c2-4954-84ad-568d97b1b254" containerName="istio-proxy"
Jul 10 04:06:39.161399 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.161373    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8f57935-fd31-47bd-80a4-f2c0a21f98d9" containerName="revoke-keys"
Jul 10 04:06:39.161399 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.161384    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a6ddd52-8bcc-429a-bf11-500d92627c78" containerName="maas-api"
Jul 10 04:06:39.166133 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.166107    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.168785 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.168764    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-010d76e9-openshift-default-dockercfg-mkxrr\""
Jul 10 04:06:39.169067 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.169049    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 04:06:39.178188 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.178160    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"]
Jul 10 04:06:39.188093 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.188068    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:39.286082 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.286047    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:39.288157 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.288128    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f05c386-1681-4e75-aab2-845889677e12-istiod-ca-cert\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.288321 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.288169    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f05c386-1681-4e75-aab2-845889677e12-istio-podinfo\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.288321 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.288196    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-credential-socket\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.288321 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.288271    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-workload-socket\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.288321 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.288306    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-workload-certs\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.288548 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.288329    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqv2x\" (UniqueName: \"kubernetes.io/projected/4f05c386-1681-4e75-aab2-845889677e12-kube-api-access-jqv2x\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.288548 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.288367    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-istio-data\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.288548 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.288385    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f05c386-1681-4e75-aab2-845889677e12-istio-token\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.288548 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.288425    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-istio-envoy\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389101 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389067    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f05c386-1681-4e75-aab2-845889677e12-istiod-ca-cert\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389278 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389124    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f05c386-1681-4e75-aab2-845889677e12-istio-podinfo\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389278 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389163    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-credential-socket\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389278 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389199    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-workload-socket\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389278 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389229    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-workload-certs\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389278 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389253    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqv2x\" (UniqueName: \"kubernetes.io/projected/4f05c386-1681-4e75-aab2-845889677e12-kube-api-access-jqv2x\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389555 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389282    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-istio-data\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389555 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389306    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f05c386-1681-4e75-aab2-845889677e12-istio-token\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389555 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389339    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-istio-envoy\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389747 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389564    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-credential-socket\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389747 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389595    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-workload-socket\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389747 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389734    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-workload-certs\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.389983 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.389963    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-istio-data\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.390122 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.390102    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f05c386-1681-4e75-aab2-845889677e12-istiod-ca-cert\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.392010 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.391989    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-istio-envoy\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.392331 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.392306    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f05c386-1681-4e75-aab2-845889677e12-istio-podinfo\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.398262 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.398235    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqv2x\" (UniqueName: \"kubernetes.io/projected/4f05c386-1681-4e75-aab2-845889677e12-kube-api-access-jqv2x\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.398522 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.398500    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f05c386-1681-4e75-aab2-845889677e12-istio-token\") pod \"e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") " pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.479672 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.479575    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:39.654753 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.654721    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"]
Jul 10 04:06:39.656245 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:06:39.656214    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f05c386_1681_4e75_aab2_845889677e12.slice/crio-edf79ee1375f4952a11f378bd860b4d26eb692c59ac4077504ed2cd74f054f67 WatchSource:0}: Error finding container edf79ee1375f4952a11f378bd860b4d26eb692c59ac4077504ed2cd74f054f67: Status 404 returned error can't find the container with id edf79ee1375f4952a11f378bd860b4d26eb692c59ac4077504ed2cd74f054f67
Jul 10 04:06:39.658568 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.658524    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:06:39.658637 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.658615    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:06:39.658705 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.658679    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:06:39.718189 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:39.718155    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" event={"ID":"4f05c386-1681-4e75-aab2-845889677e12","Type":"ContainerStarted","Data":"edf79ee1375f4952a11f378bd860b4d26eb692c59ac4077504ed2cd74f054f67"}
Jul 10 04:06:40.725761 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:40.725677    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" event={"ID":"4f05c386-1681-4e75-aab2-845889677e12","Type":"ContainerStarted","Data":"fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b"}
Jul 10 04:06:40.746363 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:40.746316    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" podStartSLOduration=1.746301754 podStartE2EDuration="1.746301754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:06:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:06:40.743798543 +0000 UTC m=+2062.234794436" watchObservedRunningTime="2026-07-10 04:06:40.746301754 +0000 UTC m=+2062.237297627"
Jul 10 04:06:41.479966 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:41.479929    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:41.484940 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:41.484909    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:41.731676 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:41.731553    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:41.732400 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:41.732383    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:42.694346 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:42.694313    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:42.794478 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:42.794449    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:42.922621 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:42.922578    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"]
Jul 10 04:06:42.923154 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:42.923135    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="68292182-87c2-4954-84ad-568d97b1b254" containerName="istio-proxy"
Jul 10 04:06:42.926485 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:42.926463    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:42.928830 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:42.928806    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 04:06:42.928931 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:42.928863    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-010d76e9-serving-cert\""
Jul 10 04:06:42.928931 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:42.928923    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 04:06:42.929056 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:42.929042    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6g9gw\""
Jul 10 04:06:42.936860 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:42.936837    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"]
Jul 10 04:06:42.953152 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:42.953077    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:43.024076 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.024028    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ac21e53-5cca-4748-b583-dab02ee5f0b8-maas-api-tls\") pod \"maas-api-e2e-mt-010d76e9-5f469f467-wxbtp\" (UID: \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:43.024268 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.024088    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac21e53-5cca-4748-b583-dab02ee5f0b8-openshift-service-ca\") pod \"maas-api-e2e-mt-010d76e9-5f469f467-wxbtp\" (UID: \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:43.024268 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.024191    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78hvv\" (UniqueName: \"kubernetes.io/projected/4ac21e53-5cca-4748-b583-dab02ee5f0b8-kube-api-access-78hvv\") pod \"maas-api-e2e-mt-010d76e9-5f469f467-wxbtp\" (UID: \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:43.088346 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.088314    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:43.125567 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.125528    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac21e53-5cca-4748-b583-dab02ee5f0b8-openshift-service-ca\") pod \"maas-api-e2e-mt-010d76e9-5f469f467-wxbtp\" (UID: \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:43.125772 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.125594    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78hvv\" (UniqueName: \"kubernetes.io/projected/4ac21e53-5cca-4748-b583-dab02ee5f0b8-kube-api-access-78hvv\") pod \"maas-api-e2e-mt-010d76e9-5f469f467-wxbtp\" (UID: \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:43.125772 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.125739    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ac21e53-5cca-4748-b583-dab02ee5f0b8-maas-api-tls\") pod \"maas-api-e2e-mt-010d76e9-5f469f467-wxbtp\" (UID: \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:43.126282 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.126258    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac21e53-5cca-4748-b583-dab02ee5f0b8-openshift-service-ca\") pod \"maas-api-e2e-mt-010d76e9-5f469f467-wxbtp\" (UID: \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:43.128069 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.128046    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ac21e53-5cca-4748-b583-dab02ee5f0b8-maas-api-tls\") pod \"maas-api-e2e-mt-010d76e9-5f469f467-wxbtp\" (UID: \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:43.133114 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.133094    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78hvv\" (UniqueName: \"kubernetes.io/projected/4ac21e53-5cca-4748-b583-dab02ee5f0b8-kube-api-access-78hvv\") pod \"maas-api-e2e-mt-010d76e9-5f469f467-wxbtp\" (UID: \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:43.241294 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.241199    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:43.380366 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.380339    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"]
Jul 10 04:06:43.381486 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:06:43.381457    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ac21e53_5cca_4748_b583_dab02ee5f0b8.slice/crio-d0a4d25de5339af7cd976edd54e43fdcf9f7b503a62f4b03cf7a272f4bceabb2 WatchSource:0}: Error finding container d0a4d25de5339af7cd976edd54e43fdcf9f7b503a62f4b03cf7a272f4bceabb2: Status 404 returned error can't find the container with id d0a4d25de5339af7cd976edd54e43fdcf9f7b503a62f4b03cf7a272f4bceabb2
Jul 10 04:06:43.546122 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.546036    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:43.642925 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.642891    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:43.743410 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:43.743381    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp" event={"ID":"4ac21e53-5cca-4748-b583-dab02ee5f0b8","Type":"ContainerStarted","Data":"d0a4d25de5339af7cd976edd54e43fdcf9f7b503a62f4b03cf7a272f4bceabb2"}
Jul 10 04:06:44.750368 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:44.750325    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp" event={"ID":"4ac21e53-5cca-4748-b583-dab02ee5f0b8","Type":"ContainerStarted","Data":"26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9"}
Jul 10 04:06:44.750780 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:44.750398    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:44.772372 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:44.772316    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp" podStartSLOduration=2.438939985 podStartE2EDuration="2.77229576s" totalImagesPullingTime="333.355775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:06:44.770110119 +0000 UTC m=+2066.261105994" watchObservedRunningTime="2026-07-10 04:06:44.77229576 +0000 UTC m=+2066.263291634"
Jul 10 04:06:45.970374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:45.970343    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:47.221269 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:47.221237    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:50.000345 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:50.000313    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:06:50.761226 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:50.761197    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:06:57.313862 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:57.312702    2561 patch_prober.go:28] interesting pod/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": read tcp 10.132.0.2:42934->10.132.0.70:15021: read: connection reset by peer" start-of-body=
Jul 10 04:06:57.313862 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:57.312775    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": read tcp 10.132.0.2:42934->10.132.0.70:15021: read: connection reset by peer"
Jul 10 04:06:57.836864 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:57.836830    2561 generic.go:356] "Generic (PLEG): container finished" podID="4f05c386-1681-4e75-aab2-845889677e12" containerID="fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b" exitCode=0
Jul 10 04:06:57.837052 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:57.836913    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" event={"ID":"4f05c386-1681-4e75-aab2-845889677e12","Type":"ContainerDied","Data":"fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b"}
Jul 10 04:06:57.837325 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:57.837312    2561 scope.go:117] "RemoveContainer" containerID="fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b"
Jul 10 04:06:57.838365 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:57.838338    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:06:57.838417 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:57.838406    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:06:57.838458 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:57.838441    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:06:58.844466 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:58.844432    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" event={"ID":"4f05c386-1681-4e75-aab2-845889677e12","Type":"ContainerStarted","Data":"d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8"}
Jul 10 04:06:59.480875 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:59.480828    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:06:59.481115 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:59.481089    2561 patch_prober.go:28] interesting pod/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 10 04:06:59.481192 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:06:59.481150    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 10 04:07:00.481107 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:00.481066    2561 patch_prober.go:28] interesting pod/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 10 04:07:00.481725 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:00.481692    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 10 04:07:01.480135 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:01.480096    2561 patch_prober.go:28] interesting pod/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 10 04:07:01.480465 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:01.480431    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 10 04:07:02.480515 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:02.480471    2561 patch_prober.go:28] interesting pod/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 10 04:07:02.481046 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:02.480544    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 10 04:07:03.483296 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:03.483260    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:07:03.483804 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:03.483667    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:07:03.484402 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:03.484383    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:07:10.239169 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:10.239129    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt"]
Jul 10 04:07:10.243908 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:10.243888    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt"
Jul 10 04:07:10.246239 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:10.246200    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:07:10.248357 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:10.248330    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt"]
Jul 10 04:07:10.271637 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:10.271611    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7064bb2-f33e-40d1-9590-96c63a285dc6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-010d76e9-n74qt\" (UID: \"e7064bb2-f33e-40d1-9590-96c63a285dc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt"
Jul 10 04:07:10.373061 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:10.373022    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7064bb2-f33e-40d1-9590-96c63a285dc6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-010d76e9-n74qt\" (UID: \"e7064bb2-f33e-40d1-9590-96c63a285dc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt"
Jul 10 04:07:10.373754 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:10.373729    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7064bb2-f33e-40d1-9590-96c63a285dc6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-010d76e9-n74qt\" (UID: \"e7064bb2-f33e-40d1-9590-96c63a285dc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt"
Jul 10 04:07:10.555906 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:10.555818    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt"
Jul 10 04:07:10.696446 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:10.696416    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt"]
Jul 10 04:07:10.697741 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:07:10.697703    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7064bb2_f33e_40d1_9590_96c63a285dc6.slice/crio-dc74b88618884c37c262c4aa0d554bbee5513778bd762da5f9c3dfd78b3acc85 WatchSource:0}: Error finding container dc74b88618884c37c262c4aa0d554bbee5513778bd762da5f9c3dfd78b3acc85: Status 404 returned error can't find the container with id dc74b88618884c37c262c4aa0d554bbee5513778bd762da5f9c3dfd78b3acc85
Jul 10 04:07:10.921786 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:10.921751    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt" event={"ID":"e7064bb2-f33e-40d1-9590-96c63a285dc6","Type":"ContainerStarted","Data":"dc74b88618884c37c262c4aa0d554bbee5513778bd762da5f9c3dfd78b3acc85"}
Jul 10 04:07:11.929915 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:11.929880    2561 generic.go:356] "Generic (PLEG): container finished" podID="e7064bb2-f33e-40d1-9590-96c63a285dc6" containerID="af7b23a484b31f112d53382d877c26545caf5ca38a3edd985a6e5da30c5d4970" exitCode=0
Jul 10 04:07:11.929915 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:11.929916    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt" event={"ID":"e7064bb2-f33e-40d1-9590-96c63a285dc6","Type":"ContainerDied","Data":"af7b23a484b31f112d53382d877c26545caf5ca38a3edd985a6e5da30c5d4970"}
Jul 10 04:07:13.071686 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:13.071661    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt"
Jul 10 04:07:13.097998 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:13.097969    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7064bb2-f33e-40d1-9590-96c63a285dc6-maas-api-service-ca\") pod \"e7064bb2-f33e-40d1-9590-96c63a285dc6\" (UID: \"e7064bb2-f33e-40d1-9590-96c63a285dc6\") "
Jul 10 04:07:13.098353 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:13.098327    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7064bb2-f33e-40d1-9590-96c63a285dc6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e7064bb2-f33e-40d1-9590-96c63a285dc6" (UID: "e7064bb2-f33e-40d1-9590-96c63a285dc6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:07:13.199001 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:13.198915    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7064bb2-f33e-40d1-9590-96c63a285dc6-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:07:13.943615 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:13.943575    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt" event={"ID":"e7064bb2-f33e-40d1-9590-96c63a285dc6","Type":"ContainerDied","Data":"dc74b88618884c37c262c4aa0d554bbee5513778bd762da5f9c3dfd78b3acc85"}
Jul 10 04:07:13.943783 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:13.943630    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc74b88618884c37c262c4aa0d554bbee5513778bd762da5f9c3dfd78b3acc85"
Jul 10 04:07:13.943783 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:13.943593    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-010d76e9-n74qt"
Jul 10 04:07:19.178124 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:19.178092    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 04:07:19.186896 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:19.186872    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 04:07:20.595145 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.595110    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"]
Jul 10 04:07:20.595604 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.595350    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp" podUID="4ac21e53-5cca-4748-b583-dab02ee5f0b8" containerName="maas-api" containerID="cri-o://26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9" gracePeriod=30
Jul 10 04:07:20.621861 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.621831    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:07:20.757035 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.757000    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp" podUID="4ac21e53-5cca-4748-b583-dab02ee5f0b8" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.71:8443/health\": dial tcp 10.132.0.71:8443: connect: connection refused"
Jul 10 04:07:20.845053 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.845025    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:07:20.967347 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.967251    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ac21e53-5cca-4748-b583-dab02ee5f0b8-maas-api-tls\") pod \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\" (UID: \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\") "
Jul 10 04:07:20.967347 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.967300    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac21e53-5cca-4748-b583-dab02ee5f0b8-openshift-service-ca\") pod \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\" (UID: \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\") "
Jul 10 04:07:20.967536 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.967367    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78hvv\" (UniqueName: \"kubernetes.io/projected/4ac21e53-5cca-4748-b583-dab02ee5f0b8-kube-api-access-78hvv\") pod \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\" (UID: \"4ac21e53-5cca-4748-b583-dab02ee5f0b8\") "
Jul 10 04:07:20.967713 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.967689    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ac21e53-5cca-4748-b583-dab02ee5f0b8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4ac21e53-5cca-4748-b583-dab02ee5f0b8" (UID: "4ac21e53-5cca-4748-b583-dab02ee5f0b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:07:20.969470 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.969448    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ac21e53-5cca-4748-b583-dab02ee5f0b8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4ac21e53-5cca-4748-b583-dab02ee5f0b8" (UID: "4ac21e53-5cca-4748-b583-dab02ee5f0b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:07:20.969538 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.969519    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ac21e53-5cca-4748-b583-dab02ee5f0b8-kube-api-access-78hvv" (OuterVolumeSpecName: "kube-api-access-78hvv") pod "4ac21e53-5cca-4748-b583-dab02ee5f0b8" (UID: "4ac21e53-5cca-4748-b583-dab02ee5f0b8"). InnerVolumeSpecName "kube-api-access-78hvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:07:20.985355 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.985324    2561 generic.go:356] "Generic (PLEG): container finished" podID="4ac21e53-5cca-4748-b583-dab02ee5f0b8" containerID="26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9" exitCode=0
Jul 10 04:07:20.985507 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.985372    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"
Jul 10 04:07:20.985507 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.985405    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp" event={"ID":"4ac21e53-5cca-4748-b583-dab02ee5f0b8","Type":"ContainerDied","Data":"26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9"}
Jul 10 04:07:20.985507 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.985466    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp" event={"ID":"4ac21e53-5cca-4748-b583-dab02ee5f0b8","Type":"ContainerDied","Data":"d0a4d25de5339af7cd976edd54e43fdcf9f7b503a62f4b03cf7a272f4bceabb2"}
Jul 10 04:07:20.985507 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.985488    2561 scope.go:117] "RemoveContainer" containerID="26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9"
Jul 10 04:07:20.999802 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:20.999779    2561 scope.go:117] "RemoveContainer" containerID="26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9"
Jul 10 04:07:21.000103 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:07:21.000071    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9\": container with ID starting with 26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9 not found: ID does not exist" containerID="26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9"
Jul 10 04:07:21.000235 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:21.000111    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9"} err="failed to get container status \"26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9\": rpc error: code = NotFound desc = could not find container \"26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9\": container with ID starting with 26f62e7b68db1fda5f90caa4828a2e7d03d9352ec11cdc35cebfce85e88e89c9 not found: ID does not exist"
Jul 10 04:07:21.011750 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:21.011705    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"]
Jul 10 04:07:21.015272 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:21.015245    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-010d76e9-5f469f467-wxbtp"]
Jul 10 04:07:21.068186 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:21.068151    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ac21e53-5cca-4748-b583-dab02ee5f0b8-maas-api-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:07:21.068186 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:21.068178    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac21e53-5cca-4748-b583-dab02ee5f0b8-openshift-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:07:21.068186 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:21.068188    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78hvv\" (UniqueName: \"kubernetes.io/projected/4ac21e53-5cca-4748-b583-dab02ee5f0b8-kube-api-access-78hvv\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:07:21.431346 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:21.431313    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:07:22.995053 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:22.995022    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ac21e53-5cca-4748-b583-dab02ee5f0b8" path="/var/lib/kubelet/pods/4ac21e53-5cca-4748-b583-dab02ee5f0b8/volumes"
Jul 10 04:07:25.314901 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:25.314854    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:07:30.337860 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:30.337823    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:07:44.703083 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:44.703013    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:07:57.733260 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:57.733227    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:07:57.853373 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:07:57.853340    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:06.667864 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:06.667824    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:07.214455 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:07.214416    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:11.995228 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:11.995140    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:12.096856 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:12.096812    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:12.612542 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:12.612510    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:12.678399 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:12.678371    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:17.611069 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:17.611036    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"]
Jul 10 04:08:17.611504 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:17.611260    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy" containerID="cri-o://d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8" gracePeriod=30
Jul 10 04:08:17.630540 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:17.630509    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:17.880250 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:17.880174    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:18.486416 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:18.486376    2561 patch_prober.go:28] interesting pod/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 04:08:18.486581 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:18.486441    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 04:08:18.944117 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:18.944087    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:19.043223 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:19.043189    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:19.155052 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:19.155020    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:19.267277 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:19.267198    2561 scope.go:117] "RemoveContainer" containerID="1c0fba62fc9450fed4af1dfabbe5a3ccaf93db5088ad1d976a75dd362a70ee52"
Jul 10 04:08:19.884089 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:19.884050    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:19.981356 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:19.981311    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:20.084418 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:20.084379    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:21.521364 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:21.521333    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:22.901402 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:22.901371    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:08:23.012632 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.012601    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f05c386-1681-4e75-aab2-845889677e12-istio-token\") pod \"4f05c386-1681-4e75-aab2-845889677e12\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") "
Jul 10 04:08:23.012823 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.012642    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-istio-envoy\") pod \"4f05c386-1681-4e75-aab2-845889677e12\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") "
Jul 10 04:08:23.012823 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.012686    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqv2x\" (UniqueName: \"kubernetes.io/projected/4f05c386-1681-4e75-aab2-845889677e12-kube-api-access-jqv2x\") pod \"4f05c386-1681-4e75-aab2-845889677e12\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") "
Jul 10 04:08:23.012823 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.012725    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f05c386-1681-4e75-aab2-845889677e12-istio-podinfo\") pod \"4f05c386-1681-4e75-aab2-845889677e12\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") "
Jul 10 04:08:23.012823 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.012752    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-workload-socket\") pod \"4f05c386-1681-4e75-aab2-845889677e12\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") "
Jul 10 04:08:23.013051 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.013015    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-credential-socket\") pod \"4f05c386-1681-4e75-aab2-845889677e12\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") "
Jul 10 04:08:23.013110 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.013076    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4f05c386-1681-4e75-aab2-845889677e12" (UID: "4f05c386-1681-4e75-aab2-845889677e12"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:08:23.013185 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.013111    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-istio-data\") pod \"4f05c386-1681-4e75-aab2-845889677e12\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") "
Jul 10 04:08:23.013185 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.013158    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-workload-certs\") pod \"4f05c386-1681-4e75-aab2-845889677e12\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") "
Jul 10 04:08:23.013449 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.013204    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f05c386-1681-4e75-aab2-845889677e12-istiod-ca-cert\") pod \"4f05c386-1681-4e75-aab2-845889677e12\" (UID: \"4f05c386-1681-4e75-aab2-845889677e12\") "
Jul 10 04:08:23.013449 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.013282    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4f05c386-1681-4e75-aab2-845889677e12" (UID: "4f05c386-1681-4e75-aab2-845889677e12"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:08:23.013559 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.013547    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-workload-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:08:23.013622 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.013567    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-credential-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:08:23.013808 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.013790    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4f05c386-1681-4e75-aab2-845889677e12" (UID: "4f05c386-1681-4e75-aab2-845889677e12"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:08:23.014421 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.014389    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-istio-data" (OuterVolumeSpecName: "istio-data") pod "4f05c386-1681-4e75-aab2-845889677e12" (UID: "4f05c386-1681-4e75-aab2-845889677e12"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:08:23.014701 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.014677    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f05c386-1681-4e75-aab2-845889677e12-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4f05c386-1681-4e75-aab2-845889677e12" (UID: "4f05c386-1681-4e75-aab2-845889677e12"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:08:23.015171 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.015151    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f05c386-1681-4e75-aab2-845889677e12-kube-api-access-jqv2x" (OuterVolumeSpecName: "kube-api-access-jqv2x") pod "4f05c386-1681-4e75-aab2-845889677e12" (UID: "4f05c386-1681-4e75-aab2-845889677e12"). InnerVolumeSpecName "kube-api-access-jqv2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:08:23.015278 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.015196    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4f05c386-1681-4e75-aab2-845889677e12-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4f05c386-1681-4e75-aab2-845889677e12" (UID: "4f05c386-1681-4e75-aab2-845889677e12"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 04:08:23.015340 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.015304    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4f05c386-1681-4e75-aab2-845889677e12" (UID: "4f05c386-1681-4e75-aab2-845889677e12"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:08:23.015413 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.015395    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f05c386-1681-4e75-aab2-845889677e12-istio-token" (OuterVolumeSpecName: "istio-token") pod "4f05c386-1681-4e75-aab2-845889677e12" (UID: "4f05c386-1681-4e75-aab2-845889677e12"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:08:23.115121 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.115076    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-istio-envoy\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:08:23.115121 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.115113    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqv2x\" (UniqueName: \"kubernetes.io/projected/4f05c386-1681-4e75-aab2-845889677e12-kube-api-access-jqv2x\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:08:23.115121 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.115128    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f05c386-1681-4e75-aab2-845889677e12-istio-podinfo\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:08:23.115374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.115140    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-istio-data\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:08:23.115374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.115153    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f05c386-1681-4e75-aab2-845889677e12-workload-certs\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:08:23.115374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.115165    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f05c386-1681-4e75-aab2-845889677e12-istiod-ca-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:08:23.115374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.115176    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f05c386-1681-4e75-aab2-845889677e12-istio-token\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:08:23.343349 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.343310    2561 generic.go:356] "Generic (PLEG): container finished" podID="4f05c386-1681-4e75-aab2-845889677e12" containerID="d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8" exitCode=0
Jul 10 04:08:23.343542 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.343341    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" event={"ID":"4f05c386-1681-4e75-aab2-845889677e12","Type":"ContainerDied","Data":"d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8"}
Jul 10 04:08:23.343542 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.343411    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p" event={"ID":"4f05c386-1681-4e75-aab2-845889677e12","Type":"ContainerDied","Data":"edf79ee1375f4952a11f378bd860b4d26eb692c59ac4077504ed2cd74f054f67"}
Jul 10 04:08:23.343542 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.343355    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"
Jul 10 04:08:23.343542 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.343441    2561 scope.go:117] "RemoveContainer" containerID="d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8"
Jul 10 04:08:23.354317 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.354300    2561 scope.go:117] "RemoveContainer" containerID="fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b"
Jul 10 04:08:23.364556 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.364525    2561 scope.go:117] "RemoveContainer" containerID="d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8"
Jul 10 04:08:23.364921 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:08:23.364894    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8\": container with ID starting with d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8 not found: ID does not exist" containerID="d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8"
Jul 10 04:08:23.364999 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.364930    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8"} err="failed to get container status \"d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8\": rpc error: code = NotFound desc = could not find container \"d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8\": container with ID starting with d7403b0277eaee5a87debe3752d9c5bcc844f6115bdd9fe13e139dafc6c710b8 not found: ID does not exist"
Jul 10 04:08:23.364999 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.364953    2561 scope.go:117] "RemoveContainer" containerID="fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b"
Jul 10 04:08:23.365555 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:08:23.365314    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b\": container with ID starting with fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b not found: ID does not exist" containerID="fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b"
Jul 10 04:08:23.365555 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.365347    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b"} err="failed to get container status \"fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b\": rpc error: code = NotFound desc = could not find container \"fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b\": container with ID starting with fa482d43d85fca2403db166c282a652e2f8412802c90aa0595159e7c7776b05b not found: ID does not exist"
Jul 10 04:08:23.369547 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.369523    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"]
Jul 10 04:08:23.373759 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:23.373735    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-010d76e9-openshift-default-5974fdf75-m6r7p"]
Jul 10 04:08:24.993851 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:24.993818    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f05c386-1681-4e75-aab2-845889677e12" path="/var/lib/kubelet/pods/4f05c386-1681-4e75-aab2-845889677e12/volumes"
Jul 10 04:08:27.069397 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:27.069361    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:27.132506 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:27.132459    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:27.763607 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:27.763555    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:33.411982 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:33.411947    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:33.519891 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:33.519842    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:33.612395 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:33.612346    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:44.171997 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:44.171958    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:08:44.523883 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:08:44.523806    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:05.535852 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:05.535811    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:11.138485 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.138444    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:11.197086 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.197047    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:11.518684 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.518565    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:11.817921 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.817848    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"]
Jul 10 04:09:11.818238 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.818224    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ac21e53-5cca-4748-b583-dab02ee5f0b8" containerName="maas-api"
Jul 10 04:09:11.818282 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.818239    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ac21e53-5cca-4748-b583-dab02ee5f0b8" containerName="maas-api"
Jul 10 04:09:11.818282 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.818253    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy"
Jul 10 04:09:11.818282 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.818258    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy"
Jul 10 04:09:11.818374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.818269    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7064bb2-f33e-40d1-9590-96c63a285dc6" containerName="revoke-keys"
Jul 10 04:09:11.818374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.818293    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7064bb2-f33e-40d1-9590-96c63a285dc6" containerName="revoke-keys"
Jul 10 04:09:11.818374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.818303    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy"
Jul 10 04:09:11.818374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.818308    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy"
Jul 10 04:09:11.818374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.818363    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7064bb2-f33e-40d1-9590-96c63a285dc6" containerName="revoke-keys"
Jul 10 04:09:11.818374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.818370    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy"
Jul 10 04:09:11.818562 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.818377    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f05c386-1681-4e75-aab2-845889677e12" containerName="istio-proxy"
Jul 10 04:09:11.818562 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.818384    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ac21e53-5cca-4748-b583-dab02ee5f0b8" containerName="maas-api"
Jul 10 04:09:11.821500 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.821484    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:11.824051 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.824027    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 04:09:11.824051 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.824048    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-72703e-openshift-default-dockercfg-pqgww\""
Jul 10 04:09:11.832877 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.832853    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"]
Jul 10 04:09:11.862538 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.862510    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:11.955714 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.955637    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-envoy\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:11.955897 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.955764    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-workload-certs\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:11.955897 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.955795    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-podinfo\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:11.955897 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.955820    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dcf934e-658f-46b9-a40c-6082992bda1c-istiod-ca-cert\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:11.955897 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.955844    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpmcc\" (UniqueName: \"kubernetes.io/projected/1dcf934e-658f-46b9-a40c-6082992bda1c-kube-api-access-cpmcc\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:11.956039 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.955898    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-credential-socket\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:11.956039 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.955937    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-token\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:11.956039 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.955959    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-data\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:11.956039 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:11.955976    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-workload-socket\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.056556 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.056522    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-envoy\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.056752 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.056583    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-workload-certs\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.056752 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.056603    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-podinfo\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.056752 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.056627    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dcf934e-658f-46b9-a40c-6082992bda1c-istiod-ca-cert\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.056752 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.056645    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpmcc\" (UniqueName: \"kubernetes.io/projected/1dcf934e-658f-46b9-a40c-6082992bda1c-kube-api-access-cpmcc\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.056981 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.056830    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-credential-socket\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.056981 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.056916    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-token\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.056981 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.056944    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-data\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.056981 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.056973    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-workload-socket\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.057185 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.057065    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-workload-certs\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.057278 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.057243    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-credential-socket\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.057370 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.057271    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-data\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.057370 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.057323    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-workload-socket\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.057455 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.057436    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dcf934e-658f-46b9-a40c-6082992bda1c-istiod-ca-cert\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.058883 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.058855    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-envoy\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.059677 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.059628    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-podinfo\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.065057 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.065037    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpmcc\" (UniqueName: \"kubernetes.io/projected/1dcf934e-658f-46b9-a40c-6082992bda1c-kube-api-access-cpmcc\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.065153 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.065061    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-token\") pod \"e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") " pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.133027 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.132988    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:12.281049 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.281021    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"]
Jul 10 04:09:12.283201 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:09:12.283163    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dcf934e_658f_46b9_a40c_6082992bda1c.slice/crio-4533a1b98af97c31b16131b47bd4b5a6cab1db3218174e2cd32ee3e38095d8c0 WatchSource:0}: Error finding container 4533a1b98af97c31b16131b47bd4b5a6cab1db3218174e2cd32ee3e38095d8c0: Status 404 returned error can't find the container with id 4533a1b98af97c31b16131b47bd4b5a6cab1db3218174e2cd32ee3e38095d8c0
Jul 10 04:09:12.286402 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.286351    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:09:12.286527 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.286501    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:09:12.286602 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.286586    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:09:12.622626 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.622587    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" event={"ID":"1dcf934e-658f-46b9-a40c-6082992bda1c","Type":"ContainerStarted","Data":"ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58"}
Jul 10 04:09:12.622843 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.622686    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" event={"ID":"1dcf934e-658f-46b9-a40c-6082992bda1c","Type":"ContainerStarted","Data":"4533a1b98af97c31b16131b47bd4b5a6cab1db3218174e2cd32ee3e38095d8c0"}
Jul 10 04:09:12.640920 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.640864    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" podStartSLOduration=1.640846105 podStartE2EDuration="1.640846105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:09:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:09:12.639295165 +0000 UTC m=+2214.130291043" watchObservedRunningTime="2026-07-10 04:09:12.640846105 +0000 UTC m=+2214.131841998"
Jul 10 04:09:12.684560 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:12.684521    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:13.133366 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:13.133325    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:13.138630 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:13.138606    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:13.628512 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:13.628474    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:13.629381 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:13.629363    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:14.601658 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.601609    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:14.697099 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.697063    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:14.842172 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.842139    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"]
Jul 10 04:09:14.846408 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.846381    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:14.848773 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.848749    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 04:09:14.848925 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.848819    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 04:09:14.849608 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.849585    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-72703e-serving-cert\""
Jul 10 04:09:14.849608 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.849604    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6g9gw\""
Jul 10 04:09:14.855929 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.855626    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"]
Jul 10 04:09:14.874631 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.874599    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:14.975390 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.975355    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:14.983537 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.983498    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3deee8a-f63f-478b-a91a-a2b5197f20cc-openshift-service-ca\") pod \"maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8\" (UID: \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:14.983705 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.983545    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3deee8a-f63f-478b-a91a-a2b5197f20cc-maas-api-tls\") pod \"maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8\" (UID: \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:14.983705 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:14.983663    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkr7h\" (UniqueName: \"kubernetes.io/projected/a3deee8a-f63f-478b-a91a-a2b5197f20cc-kube-api-access-vkr7h\") pod \"maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8\" (UID: \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:15.084950 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:15.084907    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3deee8a-f63f-478b-a91a-a2b5197f20cc-openshift-service-ca\") pod \"maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8\" (UID: \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:15.085148 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:15.084966    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3deee8a-f63f-478b-a91a-a2b5197f20cc-maas-api-tls\") pod \"maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8\" (UID: \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:15.085148 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:15.085061    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkr7h\" (UniqueName: \"kubernetes.io/projected/a3deee8a-f63f-478b-a91a-a2b5197f20cc-kube-api-access-vkr7h\") pod \"maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8\" (UID: \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:15.085592 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:15.085569    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3deee8a-f63f-478b-a91a-a2b5197f20cc-openshift-service-ca\") pod \"maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8\" (UID: \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:15.087476 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:15.087455    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3deee8a-f63f-478b-a91a-a2b5197f20cc-maas-api-tls\") pod \"maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8\" (UID: \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:15.094056 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:15.094031    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkr7h\" (UniqueName: \"kubernetes.io/projected/a3deee8a-f63f-478b-a91a-a2b5197f20cc-kube-api-access-vkr7h\") pod \"maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8\" (UID: \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:15.159913 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:15.159818    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:15.322694 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:15.322668    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"]
Jul 10 04:09:15.325301 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:09:15.325274    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3deee8a_f63f_478b_a91a_a2b5197f20cc.slice/crio-e071667a16601219e6cbd66b219521de410f754f5f54f0923022262bdee949bf WatchSource:0}: Error finding container e071667a16601219e6cbd66b219521de410f754f5f54f0923022262bdee949bf: Status 404 returned error can't find the container with id e071667a16601219e6cbd66b219521de410f754f5f54f0923022262bdee949bf
Jul 10 04:09:15.443801 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:15.443674    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:15.546197 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:15.546164    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:15.641010 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:15.640955    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8" event={"ID":"a3deee8a-f63f-478b-a91a-a2b5197f20cc","Type":"ContainerStarted","Data":"e071667a16601219e6cbd66b219521de410f754f5f54f0923022262bdee949bf"}
Jul 10 04:09:16.650164 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:16.650128    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8" event={"ID":"a3deee8a-f63f-478b-a91a-a2b5197f20cc","Type":"ContainerStarted","Data":"96d8ec3c67f5c4f2b3145661ba5a0a629a1550a3487fa19918759205fd178e08"}
Jul 10 04:09:16.650669 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:16.650268    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:16.674777 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:16.674719    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8" podStartSLOduration=2.200720945 podStartE2EDuration="2.674701709s" totalImagesPullingTime="473.980764ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:09:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:09:16.672776402 +0000 UTC m=+2218.163772302" watchObservedRunningTime="2026-07-10 04:09:16.674701709 +0000 UTC m=+2218.165697582"
Jul 10 04:09:19.132778 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:19.132744    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:19.293577 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:19.293549    2561 scope.go:117] "RemoveContainer" containerID="f27db778996a00edd8f19c5df451ba07033fcd51e5415f3473e82db397b6101f"
Jul 10 04:09:19.487447 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:19.487340    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:19.856412 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:19.856370    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:20.156363 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:20.156277    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:22.661822 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:22.661791    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:09:22.840575 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:22.840531    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:22.898436 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:22.898398    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"]
Jul 10 04:09:22.903461 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:22.903434    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:22.905814 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:22.905791    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-7fe142-serving-cert\""
Jul 10 04:09:22.915207 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:22.915175    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"]
Jul 10 04:09:22.916785 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:22.916754    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:22.958917 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:22.958880    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8d905d5-8ef7-4ded-bb22-70d053c08acb-maas-api-tls\") pod \"maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx\" (UID: \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:22.959069 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:22.958947    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b697p\" (UniqueName: \"kubernetes.io/projected/a8d905d5-8ef7-4ded-bb22-70d053c08acb-kube-api-access-b697p\") pod \"maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx\" (UID: \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:22.959069 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:22.958997    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d905d5-8ef7-4ded-bb22-70d053c08acb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx\" (UID: \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:22.974962 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:22.974919    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:23.059519 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:23.059484    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d905d5-8ef7-4ded-bb22-70d053c08acb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx\" (UID: \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:23.059847 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:23.059548    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8d905d5-8ef7-4ded-bb22-70d053c08acb-maas-api-tls\") pod \"maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx\" (UID: \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:23.059847 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:23.059594    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b697p\" (UniqueName: \"kubernetes.io/projected/a8d905d5-8ef7-4ded-bb22-70d053c08acb-kube-api-access-b697p\") pod \"maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx\" (UID: \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:23.060191 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:23.060168    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d905d5-8ef7-4ded-bb22-70d053c08acb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx\" (UID: \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:23.062130 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:23.062105    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8d905d5-8ef7-4ded-bb22-70d053c08acb-maas-api-tls\") pod \"maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx\" (UID: \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:23.068086 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:23.068062    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b697p\" (UniqueName: \"kubernetes.io/projected/a8d905d5-8ef7-4ded-bb22-70d053c08acb-kube-api-access-b697p\") pod \"maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx\" (UID: \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:23.075434 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:23.075414    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:23.217769 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:23.217678    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:23.358219 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:23.358184    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"]
Jul 10 04:09:23.359881 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:09:23.359841    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d905d5_8ef7_4ded_bb22_70d053c08acb.slice/crio-e239251121b15280f270c62142222e4ea73edfaee69da095fb614720f020140f WatchSource:0}: Error finding container e239251121b15280f270c62142222e4ea73edfaee69da095fb614720f020140f: Status 404 returned error can't find the container with id e239251121b15280f270c62142222e4ea73edfaee69da095fb614720f020140f
Jul 10 04:09:23.388226 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:23.388195    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:23.489847 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:23.489753    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:23.691678 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:23.691620    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx" event={"ID":"a8d905d5-8ef7-4ded-bb22-70d053c08acb","Type":"ContainerStarted","Data":"e239251121b15280f270c62142222e4ea73edfaee69da095fb614720f020140f"}
Jul 10 04:09:24.704606 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:24.704564    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx" event={"ID":"a8d905d5-8ef7-4ded-bb22-70d053c08acb","Type":"ContainerStarted","Data":"07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1"}
Jul 10 04:09:24.705115 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:24.704643    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:24.720666 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:24.720611    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx" podStartSLOduration=2.263026796 podStartE2EDuration="2.720595481s" totalImagesPullingTime="457.568685ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:09:24.719750752 +0000 UTC m=+2226.210746626" watchObservedRunningTime="2026-07-10 04:09:24.720595481 +0000 UTC m=+2226.211591355"
Jul 10 04:09:27.368040 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:27.367999    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:29.669895 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:29.669847    2561 patch_prober.go:28] interesting pod/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 04:09:29.670416 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:29.669926    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 04:09:30.717128 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:30.717090    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:09:31.747864 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:31.747827    2561 generic.go:356] "Generic (PLEG): container finished" podID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerID="ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58" exitCode=0
Jul 10 04:09:31.748338 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:31.747904    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" event={"ID":"1dcf934e-658f-46b9-a40c-6082992bda1c","Type":"ContainerDied","Data":"ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58"}
Jul 10 04:09:31.748338 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:31.748279    2561 scope.go:117] "RemoveContainer" containerID="ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58"
Jul 10 04:09:31.749263 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:31.749236    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:09:31.749401 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:31.749317    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:09:31.749401 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:31.749371    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:09:32.755460 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:32.755426    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" event={"ID":"1dcf934e-658f-46b9-a40c-6082992bda1c","Type":"ContainerStarted","Data":"052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b"}
Jul 10 04:09:33.133609 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:33.133570    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:33.133896 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:33.133871    2561 patch_prober.go:28] interesting pod/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Jul 10 04:09:33.133979 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:33.133931    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Jul 10 04:09:34.133711 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:34.133642    2561 patch_prober.go:28] interesting pod/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Jul 10 04:09:34.134115 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:34.133767    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Jul 10 04:09:35.134240 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:35.134188    2561 patch_prober.go:28] interesting pod/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Jul 10 04:09:35.134634 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:35.134260    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Jul 10 04:09:36.133634 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:36.133590    2561 patch_prober.go:28] interesting pod/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Jul 10 04:09:36.133839 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:36.133679    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Jul 10 04:09:36.244067 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:36.244025    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:36.345437 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:36.345401    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:37.136870 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:37.136834    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:37.137219 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:37.137197    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:37.137753 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:37.137734    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:09:38.701793 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:38.701760    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:44.232769 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:44.232679    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:09:45.408929 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:09:45.408892    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:10:09.370985 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:09.370952    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:10:36.593068 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:36.593036    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:10:37.539856 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:37.539814    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:10:37.654517 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:37.654485    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f"]
Jul 10 04:10:37.658119 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:37.658100    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f"
Jul 10 04:10:37.660350 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:37.660330    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:10:37.665113 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:37.665089    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f"]
Jul 10 04:10:37.757108 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:37.757069    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f\" (UID: \"4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f"
Jul 10 04:10:37.857869 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:37.857833    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f\" (UID: \"4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f"
Jul 10 04:10:37.858481 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:37.858460    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f\" (UID: \"4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f"
Jul 10 04:10:37.971585 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:37.971550    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f"
Jul 10 04:10:38.106466 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:38.106440    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f"]
Jul 10 04:10:38.109635 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:10:38.109569    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c95d1a6_ba37_45e5_9e4a_26e1d8a09ad6.slice/crio-8bd62473d6d1bc48e43c76fd0e6556a9dbd566400291d6bed65026ebb2471e5a WatchSource:0}: Error finding container 8bd62473d6d1bc48e43c76fd0e6556a9dbd566400291d6bed65026ebb2471e5a: Status 404 returned error can't find the container with id 8bd62473d6d1bc48e43c76fd0e6556a9dbd566400291d6bed65026ebb2471e5a
Jul 10 04:10:38.126067 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:38.126036    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f" event={"ID":"4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6","Type":"ContainerStarted","Data":"8bd62473d6d1bc48e43c76fd0e6556a9dbd566400291d6bed65026ebb2471e5a"}
Jul 10 04:10:38.255621 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:38.255582    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:10:38.356318 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:38.356273    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:10:39.133994 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:39.133960    2561 generic.go:356] "Generic (PLEG): container finished" podID="4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6" containerID="27b91ae0afd3d915775309bb8d6415c0a01990b90d584a08c98330b57ef9e84f" exitCode=0
Jul 10 04:10:39.134399 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:39.134046    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f" event={"ID":"4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6","Type":"ContainerDied","Data":"27b91ae0afd3d915775309bb8d6415c0a01990b90d584a08c98330b57ef9e84f"}
Jul 10 04:10:40.272979 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:40.272953    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f"
Jul 10 04:10:40.382146 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:40.382092    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6-maas-api-service-ca\") pod \"4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6\" (UID: \"4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6\") "
Jul 10 04:10:40.382473 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:40.382448    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6" (UID: "4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:10:40.483881 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:40.483782    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:10:41.147527 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:41.147494    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f" event={"ID":"4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6","Type":"ContainerDied","Data":"8bd62473d6d1bc48e43c76fd0e6556a9dbd566400291d6bed65026ebb2471e5a"}
Jul 10 04:10:41.147724 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:41.147543    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bd62473d6d1bc48e43c76fd0e6556a9dbd566400291d6bed65026ebb2471e5a"
Jul 10 04:10:41.147724 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:41.147500    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f"
Jul 10 04:10:48.016755 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.016720    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"]
Jul 10 04:10:48.017253 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.016967    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8" podUID="a3deee8a-f63f-478b-a91a-a2b5197f20cc" containerName="maas-api" containerID="cri-o://96d8ec3c67f5c4f2b3145661ba5a0a629a1550a3487fa19918759205fd178e08" gracePeriod=30
Jul 10 04:10:48.039887 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.039857    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:10:48.191203 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.191169    2561 generic.go:356] "Generic (PLEG): container finished" podID="a3deee8a-f63f-478b-a91a-a2b5197f20cc" containerID="96d8ec3c67f5c4f2b3145661ba5a0a629a1550a3487fa19918759205fd178e08" exitCode=0
Jul 10 04:10:48.191367 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.191240    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8" event={"ID":"a3deee8a-f63f-478b-a91a-a2b5197f20cc","Type":"ContainerDied","Data":"96d8ec3c67f5c4f2b3145661ba5a0a629a1550a3487fa19918759205fd178e08"}
Jul 10 04:10:48.272528 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.272463    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:10:48.346970 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.346937    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3deee8a-f63f-478b-a91a-a2b5197f20cc-maas-api-tls\") pod \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\" (UID: \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\") "
Jul 10 04:10:48.347162 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.346979    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3deee8a-f63f-478b-a91a-a2b5197f20cc-openshift-service-ca\") pod \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\" (UID: \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\") "
Jul 10 04:10:48.347162 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.347003    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkr7h\" (UniqueName: \"kubernetes.io/projected/a3deee8a-f63f-478b-a91a-a2b5197f20cc-kube-api-access-vkr7h\") pod \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\" (UID: \"a3deee8a-f63f-478b-a91a-a2b5197f20cc\") "
Jul 10 04:10:48.347369 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.347343    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3deee8a-f63f-478b-a91a-a2b5197f20cc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a3deee8a-f63f-478b-a91a-a2b5197f20cc" (UID: "a3deee8a-f63f-478b-a91a-a2b5197f20cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:10:48.349205 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.349176    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3deee8a-f63f-478b-a91a-a2b5197f20cc-kube-api-access-vkr7h" (OuterVolumeSpecName: "kube-api-access-vkr7h") pod "a3deee8a-f63f-478b-a91a-a2b5197f20cc" (UID: "a3deee8a-f63f-478b-a91a-a2b5197f20cc"). InnerVolumeSpecName "kube-api-access-vkr7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:10:48.349315 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.349183    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3deee8a-f63f-478b-a91a-a2b5197f20cc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a3deee8a-f63f-478b-a91a-a2b5197f20cc" (UID: "a3deee8a-f63f-478b-a91a-a2b5197f20cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:10:48.448104 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.448072    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3deee8a-f63f-478b-a91a-a2b5197f20cc-maas-api-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:10:48.448104 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.448101    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3deee8a-f63f-478b-a91a-a2b5197f20cc-openshift-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:10:48.448306 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:48.448112    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkr7h\" (UniqueName: \"kubernetes.io/projected/a3deee8a-f63f-478b-a91a-a2b5197f20cc-kube-api-access-vkr7h\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:10:49.198029 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:49.197995    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"
Jul 10 04:10:49.198470 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:49.198050    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8" event={"ID":"a3deee8a-f63f-478b-a91a-a2b5197f20cc","Type":"ContainerDied","Data":"e071667a16601219e6cbd66b219521de410f754f5f54f0923022262bdee949bf"}
Jul 10 04:10:49.198470 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:49.198107    2561 scope.go:117] "RemoveContainer" containerID="96d8ec3c67f5c4f2b3145661ba5a0a629a1550a3487fa19918759205fd178e08"
Jul 10 04:10:49.214930 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:49.214904    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"]
Jul 10 04:10:49.223224 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:49.223199    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-72703e-5c74df64c9-7xjd8"]
Jul 10 04:10:50.994603 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:50.994561    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3deee8a-f63f-478b-a91a-a2b5197f20cc" path="/var/lib/kubelet/pods/a3deee8a-f63f-478b-a91a-a2b5197f20cc/volumes"
Jul 10 04:10:52.733624 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:52.733582    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:10:58.443282 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:58.443197    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:10:58.573175 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:10:58.573143    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:03.446424 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.446384    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"]
Jul 10 04:11:03.446929 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.446606    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy" containerID="cri-o://052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b" gracePeriod=30
Jul 10 04:11:03.466369 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.466343    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:03.713803 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.713716    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:03.827330 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.827291    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn"]
Jul 10 04:11:03.827751 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.827733    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6" containerName="revoke-keys"
Jul 10 04:11:03.827839 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.827753    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6" containerName="revoke-keys"
Jul 10 04:11:03.827839 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.827774    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3deee8a-f63f-478b-a91a-a2b5197f20cc" containerName="maas-api"
Jul 10 04:11:03.827839 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.827782    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3deee8a-f63f-478b-a91a-a2b5197f20cc" containerName="maas-api"
Jul 10 04:11:03.828001 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.827922    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6" containerName="revoke-keys"
Jul 10 04:11:03.828001 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.827938    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3deee8a-f63f-478b-a91a-a2b5197f20cc" containerName="maas-api"
Jul 10 04:11:03.831135 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.831115    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn"
Jul 10 04:11:03.833559 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.833538    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:11:03.837516 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.837489    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn"]
Jul 10 04:11:03.870779 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.870746    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8818641a-d1eb-4bd2-87c0-66105ff629cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn\" (UID: \"8818641a-d1eb-4bd2-87c0-66105ff629cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn"
Jul 10 04:11:03.972107 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.972007    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8818641a-d1eb-4bd2-87c0-66105ff629cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn\" (UID: \"8818641a-d1eb-4bd2-87c0-66105ff629cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn"
Jul 10 04:11:03.972703 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:03.972679    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8818641a-d1eb-4bd2-87c0-66105ff629cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn\" (UID: \"8818641a-d1eb-4bd2-87c0-66105ff629cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn"
Jul 10 04:11:04.142899 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:04.142861    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn"
Jul 10 04:11:04.284100 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:04.284066    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn"]
Jul 10 04:11:04.286726 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:11:04.286691    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8818641a_d1eb_4bd2_87c0_66105ff629cc.slice/crio-0407a4f16558b31d2c8941b2fee741991a103a5fd22a0abdcdd8e3d62114ee3b WatchSource:0}: Error finding container 0407a4f16558b31d2c8941b2fee741991a103a5fd22a0abdcdd8e3d62114ee3b: Status 404 returned error can't find the container with id 0407a4f16558b31d2c8941b2fee741991a103a5fd22a0abdcdd8e3d62114ee3b
Jul 10 04:11:04.310567 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:04.310541    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:04.411930 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:04.411898    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:05.298508 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:05.298414    2561 generic.go:356] "Generic (PLEG): container finished" podID="8818641a-d1eb-4bd2-87c0-66105ff629cc" containerID="38877c79fa3ce9a5b5fb1fdad3d9baddf555a9f05dea6322655a1c4bf90f6415" exitCode=0
Jul 10 04:11:05.298508 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:05.298469    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn" event={"ID":"8818641a-d1eb-4bd2-87c0-66105ff629cc","Type":"ContainerDied","Data":"38877c79fa3ce9a5b5fb1fdad3d9baddf555a9f05dea6322655a1c4bf90f6415"}
Jul 10 04:11:05.298938 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:05.298523    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn" event={"ID":"8818641a-d1eb-4bd2-87c0-66105ff629cc","Type":"ContainerStarted","Data":"0407a4f16558b31d2c8941b2fee741991a103a5fd22a0abdcdd8e3d62114ee3b"}
Jul 10 04:11:06.442317 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:06.442295    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn"
Jul 10 04:11:06.495307 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:06.495276    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8818641a-d1eb-4bd2-87c0-66105ff629cc-maas-api-service-ca\") pod \"8818641a-d1eb-4bd2-87c0-66105ff629cc\" (UID: \"8818641a-d1eb-4bd2-87c0-66105ff629cc\") "
Jul 10 04:11:06.495687 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:06.495644    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8818641a-d1eb-4bd2-87c0-66105ff629cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8818641a-d1eb-4bd2-87c0-66105ff629cc" (UID: "8818641a-d1eb-4bd2-87c0-66105ff629cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:11:06.596431 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:06.596399    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8818641a-d1eb-4bd2-87c0-66105ff629cc-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:07.139910 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:07.139875    2561 patch_prober.go:28] interesting pod/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 04:11:07.140069 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:07.139936    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 04:11:07.311121 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:07.311030    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn" event={"ID":"8818641a-d1eb-4bd2-87c0-66105ff629cc","Type":"ContainerDied","Data":"0407a4f16558b31d2c8941b2fee741991a103a5fd22a0abdcdd8e3d62114ee3b"}
Jul 10 04:11:07.311121 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:07.311054    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn"
Jul 10 04:11:07.311121 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:07.311085    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0407a4f16558b31d2c8941b2fee741991a103a5fd22a0abdcdd8e3d62114ee3b"
Jul 10 04:11:08.706327 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.706302    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:11:08.816131 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816052    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dcf934e-658f-46b9-a40c-6082992bda1c-istiod-ca-cert\") pod \"1dcf934e-658f-46b9-a40c-6082992bda1c\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") "
Jul 10 04:11:08.816131 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816086    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpmcc\" (UniqueName: \"kubernetes.io/projected/1dcf934e-658f-46b9-a40c-6082992bda1c-kube-api-access-cpmcc\") pod \"1dcf934e-658f-46b9-a40c-6082992bda1c\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") "
Jul 10 04:11:08.816131 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816130    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-credential-socket\") pod \"1dcf934e-658f-46b9-a40c-6082992bda1c\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") "
Jul 10 04:11:08.816395 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816164    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-podinfo\") pod \"1dcf934e-658f-46b9-a40c-6082992bda1c\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") "
Jul 10 04:11:08.816395 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816196    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-workload-socket\") pod \"1dcf934e-658f-46b9-a40c-6082992bda1c\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") "
Jul 10 04:11:08.816395 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816242    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-data\") pod \"1dcf934e-658f-46b9-a40c-6082992bda1c\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") "
Jul 10 04:11:08.816395 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816283    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-workload-certs\") pod \"1dcf934e-658f-46b9-a40c-6082992bda1c\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") "
Jul 10 04:11:08.816395 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816311    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-token\") pod \"1dcf934e-658f-46b9-a40c-6082992bda1c\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") "
Jul 10 04:11:08.816395 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816345    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-envoy\") pod \"1dcf934e-658f-46b9-a40c-6082992bda1c\" (UID: \"1dcf934e-658f-46b9-a40c-6082992bda1c\") "
Jul 10 04:11:08.816729 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816469    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1dcf934e-658f-46b9-a40c-6082992bda1c" (UID: "1dcf934e-658f-46b9-a40c-6082992bda1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:11:08.816729 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816596    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1dcf934e-658f-46b9-a40c-6082992bda1c" (UID: "1dcf934e-658f-46b9-a40c-6082992bda1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:11:08.816729 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816613    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1dcf934e-658f-46b9-a40c-6082992bda1c" (UID: "1dcf934e-658f-46b9-a40c-6082992bda1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:11:08.816729 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816643    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-credential-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:08.816906 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.816811    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dcf934e-658f-46b9-a40c-6082992bda1c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1dcf934e-658f-46b9-a40c-6082992bda1c" (UID: "1dcf934e-658f-46b9-a40c-6082992bda1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:11:08.817200 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.817172    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-data" (OuterVolumeSpecName: "istio-data") pod "1dcf934e-658f-46b9-a40c-6082992bda1c" (UID: "1dcf934e-658f-46b9-a40c-6082992bda1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:11:08.818469 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.818442    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1dcf934e-658f-46b9-a40c-6082992bda1c" (UID: "1dcf934e-658f-46b9-a40c-6082992bda1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 04:11:08.818576 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.818551    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dcf934e-658f-46b9-a40c-6082992bda1c-kube-api-access-cpmcc" (OuterVolumeSpecName: "kube-api-access-cpmcc") pod "1dcf934e-658f-46b9-a40c-6082992bda1c" (UID: "1dcf934e-658f-46b9-a40c-6082992bda1c"). InnerVolumeSpecName "kube-api-access-cpmcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:11:08.818617 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.818568    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1dcf934e-658f-46b9-a40c-6082992bda1c" (UID: "1dcf934e-658f-46b9-a40c-6082992bda1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:11:08.818675 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.818637    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-token" (OuterVolumeSpecName: "istio-token") pod "1dcf934e-658f-46b9-a40c-6082992bda1c" (UID: "1dcf934e-658f-46b9-a40c-6082992bda1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:11:08.917579 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.917544    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dcf934e-658f-46b9-a40c-6082992bda1c-istiod-ca-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:08.917579 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.917574    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpmcc\" (UniqueName: \"kubernetes.io/projected/1dcf934e-658f-46b9-a40c-6082992bda1c-kube-api-access-cpmcc\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:08.917579 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.917585    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-podinfo\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:08.917859 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.917593    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-workload-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:08.917859 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.917602    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-data\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:08.917859 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.917610    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-workload-certs\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:08.917859 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.917618    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-token\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:08.917859 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:08.917625    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dcf934e-658f-46b9-a40c-6082992bda1c-istio-envoy\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:09.324342 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:09.324304    2561 generic.go:356] "Generic (PLEG): container finished" podID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerID="052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b" exitCode=0
Jul 10 04:11:09.324591 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:09.324366    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"
Jul 10 04:11:09.324591 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:09.324389    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" event={"ID":"1dcf934e-658f-46b9-a40c-6082992bda1c","Type":"ContainerDied","Data":"052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b"}
Jul 10 04:11:09.324591 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:09.324471    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js" event={"ID":"1dcf934e-658f-46b9-a40c-6082992bda1c","Type":"ContainerDied","Data":"4533a1b98af97c31b16131b47bd4b5a6cab1db3218174e2cd32ee3e38095d8c0"}
Jul 10 04:11:09.324591 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:09.324494    2561 scope.go:117] "RemoveContainer" containerID="052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b"
Jul 10 04:11:09.334190 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:09.334173    2561 scope.go:117] "RemoveContainer" containerID="ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58"
Jul 10 04:11:09.341482 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:09.341446    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"]
Jul 10 04:11:09.343429 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:09.343401    2561 scope.go:117] "RemoveContainer" containerID="052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b"
Jul 10 04:11:09.343715 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:11:09.343695    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b\": container with ID starting with 052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b not found: ID does not exist" containerID="052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b"
Jul 10 04:11:09.343774 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:09.343724    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b"} err="failed to get container status \"052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b\": rpc error: code = NotFound desc = could not find container \"052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b\": container with ID starting with 052f0889c9e7b5f8389cf32c657251833f86f7be3399a053a587ff2f8614444b not found: ID does not exist"
Jul 10 04:11:09.343774 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:09.343742    2561 scope.go:117] "RemoveContainer" containerID="ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58"
Jul 10 04:11:09.343994 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:11:09.343978    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58\": container with ID starting with ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58 not found: ID does not exist" containerID="ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58"
Jul 10 04:11:09.344032 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:09.343999    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58"} err="failed to get container status \"ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58\": rpc error: code = NotFound desc = could not find container \"ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58\": container with ID starting with ff18d31d7ff6fdfe548d7e4a4f94d8bad17d31a92425d4159a4291483db31c58 not found: ID does not exist"
Jul 10 04:11:09.344747 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:09.344725    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-72703e-openshift-default-7487b45bf-jx8js"]
Jul 10 04:11:10.994764 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:10.994687    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" path="/var/lib/kubelet/pods/1dcf934e-658f-46b9-a40c-6082992bda1c/volumes"
Jul 10 04:11:14.288855 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.288810    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"]
Jul 10 04:11:14.289341 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.289086    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx" podUID="a8d905d5-8ef7-4ded-bb22-70d053c08acb" containerName="maas-api" containerID="cri-o://07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1" gracePeriod=30
Jul 10 04:11:14.309455 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.309431    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:14.537363 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.537339    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:11:14.567003 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.566934    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8d905d5-8ef7-4ded-bb22-70d053c08acb-maas-api-tls\") pod \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\" (UID: \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\") "
Jul 10 04:11:14.567003 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.566977    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d905d5-8ef7-4ded-bb22-70d053c08acb-openshift-service-ca\") pod \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\" (UID: \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\") "
Jul 10 04:11:14.567165 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.567105    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b697p\" (UniqueName: \"kubernetes.io/projected/a8d905d5-8ef7-4ded-bb22-70d053c08acb-kube-api-access-b697p\") pod \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\" (UID: \"a8d905d5-8ef7-4ded-bb22-70d053c08acb\") "
Jul 10 04:11:14.567437 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.567410    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8d905d5-8ef7-4ded-bb22-70d053c08acb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a8d905d5-8ef7-4ded-bb22-70d053c08acb" (UID: "a8d905d5-8ef7-4ded-bb22-70d053c08acb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:11:14.569220 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.569184    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8d905d5-8ef7-4ded-bb22-70d053c08acb-kube-api-access-b697p" (OuterVolumeSpecName: "kube-api-access-b697p") pod "a8d905d5-8ef7-4ded-bb22-70d053c08acb" (UID: "a8d905d5-8ef7-4ded-bb22-70d053c08acb"). InnerVolumeSpecName "kube-api-access-b697p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:11:14.569220 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.569205    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8d905d5-8ef7-4ded-bb22-70d053c08acb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a8d905d5-8ef7-4ded-bb22-70d053c08acb" (UID: "a8d905d5-8ef7-4ded-bb22-70d053c08acb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:11:14.668048 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.668017    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8d905d5-8ef7-4ded-bb22-70d053c08acb-maas-api-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:14.668048 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.668045    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d905d5-8ef7-4ded-bb22-70d053c08acb-openshift-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:14.668243 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:14.668056    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b697p\" (UniqueName: \"kubernetes.io/projected/a8d905d5-8ef7-4ded-bb22-70d053c08acb-kube-api-access-b697p\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:15.361032 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:15.360996    2561 generic.go:356] "Generic (PLEG): container finished" podID="a8d905d5-8ef7-4ded-bb22-70d053c08acb" containerID="07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1" exitCode=0
Jul 10 04:11:15.361522 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:15.361051    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"
Jul 10 04:11:15.361522 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:15.361053    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx" event={"ID":"a8d905d5-8ef7-4ded-bb22-70d053c08acb","Type":"ContainerDied","Data":"07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1"}
Jul 10 04:11:15.361522 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:15.361182    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx" event={"ID":"a8d905d5-8ef7-4ded-bb22-70d053c08acb","Type":"ContainerDied","Data":"e239251121b15280f270c62142222e4ea73edfaee69da095fb614720f020140f"}
Jul 10 04:11:15.361522 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:15.361214    2561 scope.go:117] "RemoveContainer" containerID="07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1"
Jul 10 04:11:15.370381 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:15.370363    2561 scope.go:117] "RemoveContainer" containerID="07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1"
Jul 10 04:11:15.370637 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:11:15.370619    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1\": container with ID starting with 07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1 not found: ID does not exist" containerID="07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1"
Jul 10 04:11:15.370744 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:15.370723    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1"} err="failed to get container status \"07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1\": rpc error: code = NotFound desc = could not find container \"07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1\": container with ID starting with 07b4fe398dd4536ec06d23eb509d6dab16849756163b5eaab097c479b0edc2e1 not found: ID does not exist"
Jul 10 04:11:15.378377 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:15.378357    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"]
Jul 10 04:11:15.387894 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:15.387872    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7fe142-6dd4b59d99-cpzcx"]
Jul 10 04:11:16.994613 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:16.994582    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8d905d5-8ef7-4ded-bb22-70d053c08acb" path="/var/lib/kubelet/pods/a8d905d5-8ef7-4ded-bb22-70d053c08acb/volumes"
Jul 10 04:11:18.902380 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:18.902350    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:24.693148 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:24.693113    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:24.757021 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:24.756989    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:29.622759 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:29.622725    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:29.871497 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:29.871464    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:30.225384 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.225341    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:30.529962 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.529880    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"]
Jul 10 04:11:30.530491 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.530474    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d905d5-8ef7-4ded-bb22-70d053c08acb" containerName="maas-api"
Jul 10 04:11:30.530545 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.530494    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d905d5-8ef7-4ded-bb22-70d053c08acb" containerName="maas-api"
Jul 10 04:11:30.530545 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.530519    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy"
Jul 10 04:11:30.530545 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.530528    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy"
Jul 10 04:11:30.530545 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.530539    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy"
Jul 10 04:11:30.530691 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.530548    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy"
Jul 10 04:11:30.530691 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.530562    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8818641a-d1eb-4bd2-87c0-66105ff629cc" containerName="revoke-keys"
Jul 10 04:11:30.530691 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.530571    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8818641a-d1eb-4bd2-87c0-66105ff629cc" containerName="revoke-keys"
Jul 10 04:11:30.530691 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.530677    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy"
Jul 10 04:11:30.530691 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.530689    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8818641a-d1eb-4bd2-87c0-66105ff629cc" containerName="revoke-keys"
Jul 10 04:11:30.530848 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.530700    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8d905d5-8ef7-4ded-bb22-70d053c08acb" containerName="maas-api"
Jul 10 04:11:30.534164 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.534145    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.536506 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.536483    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 04:11:30.536619 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.536482    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f48e73-openshift-default-dockercfg-qrq6w\""
Jul 10 04:11:30.544354 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.544327    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"]
Jul 10 04:11:30.554570 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.554541    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:30.615226 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.615186    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-envoy\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.615404 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.615248    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-credential-socket\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.615404 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.615293    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-workload-socket\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.615404 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.615330    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istiod-ca-cert\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.615404 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.615356    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-podinfo\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.615404 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.615382    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-token\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.615596 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.615412    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-data\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.615596 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.615438    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-workload-certs\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.615596 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.615466    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6xwm\" (UniqueName: \"kubernetes.io/projected/1908ceaf-2cf1-4cff-a470-08e24ea13d09-kube-api-access-p6xwm\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.684408 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.684365    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:30.716424 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.716374    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-envoy\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.716424 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.716432    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-credential-socket\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.716731 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.716472    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-workload-socket\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.716731 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.716496    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istiod-ca-cert\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.716731 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.716521    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-podinfo\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.716731 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.716549    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-token\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.716731 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.716583    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-data\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.716731 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.716613    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-workload-certs\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.716731 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.716639    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6xwm\" (UniqueName: \"kubernetes.io/projected/1908ceaf-2cf1-4cff-a470-08e24ea13d09-kube-api-access-p6xwm\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.717167 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.716950    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-credential-socket\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.717167 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.717028    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-workload-certs\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.717167 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.717098    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-data\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.717290 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.717229    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-workload-socket\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.717745 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.717724    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istiod-ca-cert\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.718909 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.718888    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-envoy\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.719030 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.719012    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-podinfo\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.729125 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.729093    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-token\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.729269 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.729251    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6xwm\" (UniqueName: \"kubernetes.io/projected/1908ceaf-2cf1-4cff-a470-08e24ea13d09-kube-api-access-p6xwm\") pod \"e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") " pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.847919 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.847878    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:30.992614 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:11:30.992584    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1908ceaf_2cf1_4cff_a470_08e24ea13d09.slice/crio-ddbbfe9612d2b0bdee9b3ebd5df305c4f54cc392d33dd0f184002b9a39ff9c15 WatchSource:0}: Error finding container ddbbfe9612d2b0bdee9b3ebd5df305c4f54cc392d33dd0f184002b9a39ff9c15: Status 404 returned error can't find the container with id ddbbfe9612d2b0bdee9b3ebd5df305c4f54cc392d33dd0f184002b9a39ff9c15
Jul 10 04:11:30.994677 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.994619    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:11:30.994755 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.994718    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:11:30.994797 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.994759    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:11:30.995303 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:30.995284    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"]
Jul 10 04:11:31.414485 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:31.414446    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:31.459842 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:31.459802    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" event={"ID":"1908ceaf-2cf1-4cff-a470-08e24ea13d09","Type":"ContainerStarted","Data":"dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10"}
Jul 10 04:11:31.460015 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:31.459864    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" event={"ID":"1908ceaf-2cf1-4cff-a470-08e24ea13d09","Type":"ContainerStarted","Data":"ddbbfe9612d2b0bdee9b3ebd5df305c4f54cc392d33dd0f184002b9a39ff9c15"}
Jul 10 04:11:31.471078 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:31.471035    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:31.477000 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:31.476945    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" podStartSLOduration=1.47692631 podStartE2EDuration="1.47692631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:11:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:11:31.476240827 +0000 UTC m=+2352.967236701" watchObservedRunningTime="2026-07-10 04:11:31.47692631 +0000 UTC m=+2352.967922186"
Jul 10 04:11:31.848693 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:31.848634    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:32.853509 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:32.853476    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:33.403953 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.403914    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:33.473465 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.473432    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:33.474749 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.474726    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:33.475493 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.475474    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"]
Jul 10 04:11:33.475932 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.475918    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dcf934e-658f-46b9-a40c-6082992bda1c" containerName="istio-proxy"
Jul 10 04:11:33.479077 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.479056    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:33.481196 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.481178    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f48e73-serving-cert\""
Jul 10 04:11:33.481325 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.481306    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6g9gw\""
Jul 10 04:11:33.481501 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.481379    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 04:11:33.481501 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.481219    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 04:11:33.489354 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.489334    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"]
Jul 10 04:11:33.540794 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.540756    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3918ee5-26e8-4b76-a875-13996b9c1746-openshift-service-ca\") pod \"maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z\" (UID: \"a3918ee5-26e8-4b76-a875-13996b9c1746\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:33.540941 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.540894    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3918ee5-26e8-4b76-a875-13996b9c1746-maas-api-tls\") pod \"maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z\" (UID: \"a3918ee5-26e8-4b76-a875-13996b9c1746\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:33.541039 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.541021    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xjwx\" (UniqueName: \"kubernetes.io/projected/a3918ee5-26e8-4b76-a875-13996b9c1746-kube-api-access-5xjwx\") pod \"maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z\" (UID: \"a3918ee5-26e8-4b76-a875-13996b9c1746\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:33.542376 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.542328    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:33.641488 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.641451    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:33.641691 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.641537    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xjwx\" (UniqueName: \"kubernetes.io/projected/a3918ee5-26e8-4b76-a875-13996b9c1746-kube-api-access-5xjwx\") pod \"maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z\" (UID: \"a3918ee5-26e8-4b76-a875-13996b9c1746\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:33.641691 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.641634    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3918ee5-26e8-4b76-a875-13996b9c1746-openshift-service-ca\") pod \"maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z\" (UID: \"a3918ee5-26e8-4b76-a875-13996b9c1746\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:33.641794 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.641695    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3918ee5-26e8-4b76-a875-13996b9c1746-maas-api-tls\") pod \"maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z\" (UID: \"a3918ee5-26e8-4b76-a875-13996b9c1746\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:33.642318 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.642299    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3918ee5-26e8-4b76-a875-13996b9c1746-openshift-service-ca\") pod \"maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z\" (UID: \"a3918ee5-26e8-4b76-a875-13996b9c1746\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:33.644136 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.644116    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3918ee5-26e8-4b76-a875-13996b9c1746-maas-api-tls\") pod \"maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z\" (UID: \"a3918ee5-26e8-4b76-a875-13996b9c1746\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:33.649566 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.649548    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xjwx\" (UniqueName: \"kubernetes.io/projected/a3918ee5-26e8-4b76-a875-13996b9c1746-kube-api-access-5xjwx\") pod \"maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z\" (UID: \"a3918ee5-26e8-4b76-a875-13996b9c1746\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:33.741546 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.741460    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:33.793023 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.792987    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:33.841911 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.841879    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:33.933372 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.933349    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"]
Jul 10 04:11:33.935051 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:11:33.935028    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3918ee5_26e8_4b76_a875_13996b9c1746.slice/crio-1c7011ed21870c34944ddf4e7c4beae80d29ec075447d88b10e192d56bb26fa6 WatchSource:0}: Error finding container 1c7011ed21870c34944ddf4e7c4beae80d29ec075447d88b10e192d56bb26fa6: Status 404 returned error can't find the container with id 1c7011ed21870c34944ddf4e7c4beae80d29ec075447d88b10e192d56bb26fa6
Jul 10 04:11:33.936904 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.936886    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 04:11:33.951401 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:33.951372    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:34.052837 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:34.052751    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:34.480984 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:34.480945    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z" event={"ID":"a3918ee5-26e8-4b76-a875-13996b9c1746","Type":"ContainerStarted","Data":"bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833"}
Jul 10 04:11:34.481203 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:34.481043    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z" event={"ID":"a3918ee5-26e8-4b76-a875-13996b9c1746","Type":"ContainerStarted","Data":"1c7011ed21870c34944ddf4e7c4beae80d29ec075447d88b10e192d56bb26fa6"}
Jul 10 04:11:34.481256 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:34.481233    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:34.498627 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:34.498569    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z" podStartSLOduration=1.092943923 podStartE2EDuration="1.498553397s" totalImagesPullingTime="405.609474ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:11:34.495701349 +0000 UTC m=+2355.986697224" watchObservedRunningTime="2026-07-10 04:11:34.498553397 +0000 UTC m=+2355.989549270"
Jul 10 04:11:37.965966 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:37.965924    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:38.458485 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:38.458451    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:38.763673 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:38.763563    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:39.253474 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:39.253428    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:40.501013 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:40.500980    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:11:41.517418 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.517383    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:41.586942 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.586907    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:41.597129 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.597085    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"]
Jul 10 04:11:41.601848 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.601822    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:41.605539 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.605515    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a335e2-serving-cert\""
Jul 10 04:11:41.617719 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.617693    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"]
Jul 10 04:11:41.652104 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.652076    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:41.716382 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.716346    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20380e00-a881-4e18-ad48-c58ddc336a08-openshift-service-ca\") pod \"maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:41.716590 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.716413    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20380e00-a881-4e18-ad48-c58ddc336a08-maas-api-tls\") pod \"maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:41.716590 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.716440    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pcss\" (UniqueName: \"kubernetes.io/projected/20380e00-a881-4e18-ad48-c58ddc336a08-kube-api-access-5pcss\") pod \"maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:41.817563 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.817457    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20380e00-a881-4e18-ad48-c58ddc336a08-maas-api-tls\") pod \"maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:41.817563 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.817506    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pcss\" (UniqueName: \"kubernetes.io/projected/20380e00-a881-4e18-ad48-c58ddc336a08-kube-api-access-5pcss\") pod \"maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:41.817805 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:11:41.817626    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-serving-cert: secret "maas-api-e2e-api-b-a335e2-serving-cert" not found
Jul 10 04:11:41.817805 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:11:41.817711    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20380e00-a881-4e18-ad48-c58ddc336a08-maas-api-tls podName:20380e00-a881-4e18-ad48-c58ddc336a08 nodeName:}" failed. No retries permitted until 2026-07-10 04:11:42.317694672 +0000 UTC m=+2363.808690525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/20380e00-a881-4e18-ad48-c58ddc336a08-maas-api-tls") pod "maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4" (UID: "20380e00-a881-4e18-ad48-c58ddc336a08") : secret "maas-api-e2e-api-b-a335e2-serving-cert" not found
Jul 10 04:11:41.817805 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.817638    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20380e00-a881-4e18-ad48-c58ddc336a08-openshift-service-ca\") pod \"maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:41.818385 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.818356    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20380e00-a881-4e18-ad48-c58ddc336a08-openshift-service-ca\") pod \"maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:41.828026 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.827996    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pcss\" (UniqueName: \"kubernetes.io/projected/20380e00-a881-4e18-ad48-c58ddc336a08-kube-api-access-5pcss\") pod \"maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:41.983103 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:41.983060    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:42.302272 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:42.302228    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:42.322233 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:42.322192    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20380e00-a881-4e18-ad48-c58ddc336a08-maas-api-tls\") pod \"maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:42.324793 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:42.324771    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20380e00-a881-4e18-ad48-c58ddc336a08-maas-api-tls\") pod \"maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:42.402212 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:42.402045    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:42.520818 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:42.520782    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:42.707704 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:11:42.704325    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20380e00_a881_4e18_ad48_c58ddc336a08.slice/crio-acdba1fbf399250114312db145647868207bb5f0dd26250281a5dd02fb75c72b WatchSource:0}: Error finding container acdba1fbf399250114312db145647868207bb5f0dd26250281a5dd02fb75c72b: Status 404 returned error can't find the container with id acdba1fbf399250114312db145647868207bb5f0dd26250281a5dd02fb75c72b
Jul 10 04:11:42.708729 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:42.708696    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"]
Jul 10 04:11:43.538105 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:43.538071    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4" event={"ID":"20380e00-a881-4e18-ad48-c58ddc336a08","Type":"ContainerStarted","Data":"5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234"}
Jul 10 04:11:43.538620 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:43.538130    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:43.538620 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:43.538154    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4" event={"ID":"20380e00-a881-4e18-ad48-c58ddc336a08","Type":"ContainerStarted","Data":"acdba1fbf399250114312db145647868207bb5f0dd26250281a5dd02fb75c72b"}
Jul 10 04:11:43.555992 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:43.555932    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4" podStartSLOduration=2.233675003 podStartE2EDuration="2.555913677s" totalImagesPullingTime="322.238674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:11:43.554839515 +0000 UTC m=+2365.045835412" watchObservedRunningTime="2026-07-10 04:11:43.555913677 +0000 UTC m=+2365.046909553"
Jul 10 04:11:46.039908 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:46.039871    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:47.563229 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:47.563193    2561 generic.go:356] "Generic (PLEG): container finished" podID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerID="dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10" exitCode=0
Jul 10 04:11:47.563714 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:47.563270    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" event={"ID":"1908ceaf-2cf1-4cff-a470-08e24ea13d09","Type":"ContainerDied","Data":"dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10"}
Jul 10 04:11:47.563714 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:47.563689    2561 scope.go:117] "RemoveContainer" containerID="dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10"
Jul 10 04:11:47.564696 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:47.564664    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:11:47.564808 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:47.564747    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:11:47.564808 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:47.564782    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:11:48.473678 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:48.473609    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:48.570924 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:48.570888    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" event={"ID":"1908ceaf-2cf1-4cff-a470-08e24ea13d09","Type":"ContainerStarted","Data":"830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd"}
Jul 10 04:11:48.848442 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:48.848411    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:48.849594 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:48.849568    2561 patch_prober.go:28] interesting pod/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 10 04:11:48.849738 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:48.849615    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 10 04:11:49.549790 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:49.549762    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:11:49.848956 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:49.848918    2561 patch_prober.go:28] interesting pod/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 10 04:11:49.849546 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:49.849511    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 10 04:11:50.849248 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:50.849211    2561 patch_prober.go:28] interesting pod/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 10 04:11:50.849692 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:50.849270    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 10 04:11:51.848746 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:51.848709    2561 patch_prober.go:28] interesting pod/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 10 04:11:51.848949 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:51.848773    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 10 04:11:52.870726 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:52.870694    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:52.871204 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:52.871105    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:52.871894 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:52.871874    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:11:54.213533 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:54.213495    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7"]
Jul 10 04:11:54.217555 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:54.217538    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7"
Jul 10 04:11:54.219808 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:54.219779    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:11:54.223745 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:54.223724    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7"]
Jul 10 04:11:54.238501 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:54.238474    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/418240d0-3383-4ffe-a8b7-6ccce10de838-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7\" (UID: \"418240d0-3383-4ffe-a8b7-6ccce10de838\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7"
Jul 10 04:11:54.339828 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:54.339795    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/418240d0-3383-4ffe-a8b7-6ccce10de838-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7\" (UID: \"418240d0-3383-4ffe-a8b7-6ccce10de838\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7"
Jul 10 04:11:54.340433 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:54.340414    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/418240d0-3383-4ffe-a8b7-6ccce10de838-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7\" (UID: \"418240d0-3383-4ffe-a8b7-6ccce10de838\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7"
Jul 10 04:11:54.530495 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:54.530392    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7"
Jul 10 04:11:54.650072 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:54.650021    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:54.670401 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:54.670380    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7"]
Jul 10 04:11:54.672195 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:11:54.672166    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod418240d0_3383_4ffe_a8b7_6ccce10de838.slice/crio-9257a967cbd080fc4061285f78d4c494b95f219dff2e7538bbbf8387cda396f8 WatchSource:0}: Error finding container 9257a967cbd080fc4061285f78d4c494b95f219dff2e7538bbbf8387cda396f8: Status 404 returned error can't find the container with id 9257a967cbd080fc4061285f78d4c494b95f219dff2e7538bbbf8387cda396f8
Jul 10 04:11:54.749275 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:54.749238    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:11:55.633360 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:55.633330    2561 generic.go:356] "Generic (PLEG): container finished" podID="418240d0-3383-4ffe-a8b7-6ccce10de838" containerID="458f84995ea12768bc8d332fa71be957e36122baae8b585f2c55a1e0c1d0c4c4" exitCode=0
Jul 10 04:11:55.633753 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:55.633411    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7" event={"ID":"418240d0-3383-4ffe-a8b7-6ccce10de838","Type":"ContainerDied","Data":"458f84995ea12768bc8d332fa71be957e36122baae8b585f2c55a1e0c1d0c4c4"}
Jul 10 04:11:55.633753 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:55.633490    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7" event={"ID":"418240d0-3383-4ffe-a8b7-6ccce10de838","Type":"ContainerStarted","Data":"9257a967cbd080fc4061285f78d4c494b95f219dff2e7538bbbf8387cda396f8"}
Jul 10 04:11:56.777581 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:56.777557    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7"
Jul 10 04:11:56.863530 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:56.863489    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/418240d0-3383-4ffe-a8b7-6ccce10de838-maas-api-service-ca\") pod \"418240d0-3383-4ffe-a8b7-6ccce10de838\" (UID: \"418240d0-3383-4ffe-a8b7-6ccce10de838\") "
Jul 10 04:11:56.863926 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:56.863904    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/418240d0-3383-4ffe-a8b7-6ccce10de838-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "418240d0-3383-4ffe-a8b7-6ccce10de838" (UID: "418240d0-3383-4ffe-a8b7-6ccce10de838"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:11:56.965296 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:56.965213    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/418240d0-3383-4ffe-a8b7-6ccce10de838-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:11:57.647163 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:57.647124    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7" event={"ID":"418240d0-3383-4ffe-a8b7-6ccce10de838","Type":"ContainerDied","Data":"9257a967cbd080fc4061285f78d4c494b95f219dff2e7538bbbf8387cda396f8"}
Jul 10 04:11:57.647163 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:57.647162    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7"
Jul 10 04:11:57.647163 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:11:57.647173    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9257a967cbd080fc4061285f78d4c494b95f219dff2e7538bbbf8387cda396f8"
Jul 10 04:12:04.032636 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:04.032591    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:04.130608 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:04.130573    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:04.822749 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:04.822715    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:04.924157 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:04.924124    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:05.652115 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:05.652082    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:05.751092 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:05.751064    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:05.874399 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:05.874367    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"]
Jul 10 04:12:05.874608 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:05.874579    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z" podUID="a3918ee5-26e8-4b76-a875-13996b9c1746" containerName="maas-api" containerID="cri-o://bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833" gracePeriod=30
Jul 10 04:12:05.897754 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:05.897719    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:06.121826 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.121797    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:12:06.249252 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.249157    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xjwx\" (UniqueName: \"kubernetes.io/projected/a3918ee5-26e8-4b76-a875-13996b9c1746-kube-api-access-5xjwx\") pod \"a3918ee5-26e8-4b76-a875-13996b9c1746\" (UID: \"a3918ee5-26e8-4b76-a875-13996b9c1746\") "
Jul 10 04:12:06.249252 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.249213    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3918ee5-26e8-4b76-a875-13996b9c1746-openshift-service-ca\") pod \"a3918ee5-26e8-4b76-a875-13996b9c1746\" (UID: \"a3918ee5-26e8-4b76-a875-13996b9c1746\") "
Jul 10 04:12:06.249449 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.249280    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3918ee5-26e8-4b76-a875-13996b9c1746-maas-api-tls\") pod \"a3918ee5-26e8-4b76-a875-13996b9c1746\" (UID: \"a3918ee5-26e8-4b76-a875-13996b9c1746\") "
Jul 10 04:12:06.249614 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.249587    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3918ee5-26e8-4b76-a875-13996b9c1746-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a3918ee5-26e8-4b76-a875-13996b9c1746" (UID: "a3918ee5-26e8-4b76-a875-13996b9c1746"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:12:06.251448 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.251421    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3918ee5-26e8-4b76-a875-13996b9c1746-kube-api-access-5xjwx" (OuterVolumeSpecName: "kube-api-access-5xjwx") pod "a3918ee5-26e8-4b76-a875-13996b9c1746" (UID: "a3918ee5-26e8-4b76-a875-13996b9c1746"). InnerVolumeSpecName "kube-api-access-5xjwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:12:06.251569 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.251447    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3918ee5-26e8-4b76-a875-13996b9c1746-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a3918ee5-26e8-4b76-a875-13996b9c1746" (UID: "a3918ee5-26e8-4b76-a875-13996b9c1746"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:12:06.349910 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.349873    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xjwx\" (UniqueName: \"kubernetes.io/projected/a3918ee5-26e8-4b76-a875-13996b9c1746-kube-api-access-5xjwx\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:06.349910 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.349909    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3918ee5-26e8-4b76-a875-13996b9c1746-openshift-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:06.350121 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.349922    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3918ee5-26e8-4b76-a875-13996b9c1746-maas-api-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:06.702160 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.702126    2561 generic.go:356] "Generic (PLEG): container finished" podID="a3918ee5-26e8-4b76-a875-13996b9c1746" containerID="bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833" exitCode=0
Jul 10 04:12:06.702679 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.702174    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"
Jul 10 04:12:06.702679 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.702217    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z" event={"ID":"a3918ee5-26e8-4b76-a875-13996b9c1746","Type":"ContainerDied","Data":"bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833"}
Jul 10 04:12:06.702679 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.702293    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z" event={"ID":"a3918ee5-26e8-4b76-a875-13996b9c1746","Type":"ContainerDied","Data":"1c7011ed21870c34944ddf4e7c4beae80d29ec075447d88b10e192d56bb26fa6"}
Jul 10 04:12:06.702679 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.702329    2561 scope.go:117] "RemoveContainer" containerID="bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833"
Jul 10 04:12:06.712070 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.712054    2561 scope.go:117] "RemoveContainer" containerID="bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833"
Jul 10 04:12:06.712326 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:12:06.712310    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833\": container with ID starting with bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833 not found: ID does not exist" containerID="bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833"
Jul 10 04:12:06.712411 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.712334    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833"} err="failed to get container status \"bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833\": rpc error: code = NotFound desc = could not find container \"bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833\": container with ID starting with bf87433cc819a8818d2e6024d86d4f4cd2ef0601f8e37b0f76a251e8ba3b5833 not found: ID does not exist"
Jul 10 04:12:06.728701 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.728676    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"]
Jul 10 04:12:06.731197 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.731178    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f48e73-7bf4c5877f-nbh9z"]
Jul 10 04:12:06.994035 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:06.993957    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3918ee5-26e8-4b76-a875-13996b9c1746" path="/var/lib/kubelet/pods/a3918ee5-26e8-4b76-a875-13996b9c1746/volumes"
Jul 10 04:12:09.292935 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:09.292903    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:14.996661 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:14.996613    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:15.051250 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:15.051221    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:19.214989 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:19.214958    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 04:12:19.233106 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:19.233081    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 04:12:20.006589 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.006548    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"]
Jul 10 04:12:20.007026 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.006991    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerName="istio-proxy" containerID="cri-o://830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd" gracePeriod=30
Jul 10 04:12:20.022411 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.022375    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:20.265923 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.265833    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:20.401269 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.401227    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z"]
Jul 10 04:12:20.401707 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.401688    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="418240d0-3383-4ffe-a8b7-6ccce10de838" containerName="revoke-keys"
Jul 10 04:12:20.401707 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.401708    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="418240d0-3383-4ffe-a8b7-6ccce10de838" containerName="revoke-keys"
Jul 10 04:12:20.401863 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.401725    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3918ee5-26e8-4b76-a875-13996b9c1746" containerName="maas-api"
Jul 10 04:12:20.401863 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.401733    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3918ee5-26e8-4b76-a875-13996b9c1746" containerName="maas-api"
Jul 10 04:12:20.401863 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.401833    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="418240d0-3383-4ffe-a8b7-6ccce10de838" containerName="revoke-keys"
Jul 10 04:12:20.401863 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.401847    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3918ee5-26e8-4b76-a875-13996b9c1746" containerName="maas-api"
Jul 10 04:12:20.406468 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.406444    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z"
Jul 10 04:12:20.408919 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.408895    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:12:20.411288 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.411258    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z"]
Jul 10 04:12:20.473376 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.473337    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29978b1d-4bb8-48dc-8056-1ada76651e6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z\" (UID: \"29978b1d-4bb8-48dc-8056-1ada76651e6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z"
Jul 10 04:12:20.573966 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.573924    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29978b1d-4bb8-48dc-8056-1ada76651e6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z\" (UID: \"29978b1d-4bb8-48dc-8056-1ada76651e6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z"
Jul 10 04:12:20.574566 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.574546    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29978b1d-4bb8-48dc-8056-1ada76651e6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z\" (UID: \"29978b1d-4bb8-48dc-8056-1ada76651e6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z"
Jul 10 04:12:20.718808 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.718767    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z"
Jul 10 04:12:20.858186 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:20.858155    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z"]
Jul 10 04:12:20.859358 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:12:20.859333    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29978b1d_4bb8_48dc_8056_1ada76651e6e.slice/crio-6a82c05413a37e9a188a1ef2167b2022c4a6908fa1b8fba905ad06f5785358e6 WatchSource:0}: Error finding container 6a82c05413a37e9a188a1ef2167b2022c4a6908fa1b8fba905ad06f5785358e6: Status 404 returned error can't find the container with id 6a82c05413a37e9a188a1ef2167b2022c4a6908fa1b8fba905ad06f5785358e6
Jul 10 04:12:21.798633 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:21.798598    2561 generic.go:356] "Generic (PLEG): container finished" podID="29978b1d-4bb8-48dc-8056-1ada76651e6e" containerID="b09104d5e735ce13a3d35163fcdbb421ccaad17ab4861c28c0da18fe640b32fe" exitCode=0
Jul 10 04:12:21.799070 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:21.798667    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z" event={"ID":"29978b1d-4bb8-48dc-8056-1ada76651e6e","Type":"ContainerDied","Data":"b09104d5e735ce13a3d35163fcdbb421ccaad17ab4861c28c0da18fe640b32fe"}
Jul 10 04:12:21.799070 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:21.798707    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z" event={"ID":"29978b1d-4bb8-48dc-8056-1ada76651e6e","Type":"ContainerStarted","Data":"6a82c05413a37e9a188a1ef2167b2022c4a6908fa1b8fba905ad06f5785358e6"}
Jul 10 04:12:22.873628 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:22.873594    2561 patch_prober.go:28] interesting pod/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 10 04:12:22.873974 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:22.873677    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 04:12:22.941393 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:22.941366    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z"
Jul 10 04:12:23.093334 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:23.093303    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29978b1d-4bb8-48dc-8056-1ada76651e6e-maas-api-service-ca\") pod \"29978b1d-4bb8-48dc-8056-1ada76651e6e\" (UID: \"29978b1d-4bb8-48dc-8056-1ada76651e6e\") "
Jul 10 04:12:23.093725 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:23.093704    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29978b1d-4bb8-48dc-8056-1ada76651e6e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29978b1d-4bb8-48dc-8056-1ada76651e6e" (UID: "29978b1d-4bb8-48dc-8056-1ada76651e6e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:12:23.194124 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:23.194076    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29978b1d-4bb8-48dc-8056-1ada76651e6e-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:23.812006 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:23.811977    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z"
Jul 10 04:12:23.812177 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:23.812017    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z" event={"ID":"29978b1d-4bb8-48dc-8056-1ada76651e6e","Type":"ContainerDied","Data":"6a82c05413a37e9a188a1ef2167b2022c4a6908fa1b8fba905ad06f5785358e6"}
Jul 10 04:12:23.812177 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:23.812065    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a82c05413a37e9a188a1ef2167b2022c4a6908fa1b8fba905ad06f5785358e6"
Jul 10 04:12:25.265580 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.265555    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:12:25.417534 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.417497    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-credential-socket\") pod \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") "
Jul 10 04:12:25.417754 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.417552    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6xwm\" (UniqueName: \"kubernetes.io/projected/1908ceaf-2cf1-4cff-a470-08e24ea13d09-kube-api-access-p6xwm\") pod \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") "
Jul 10 04:12:25.417754 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.417585    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istiod-ca-cert\") pod \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") "
Jul 10 04:12:25.417754 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.417672    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-envoy\") pod \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") "
Jul 10 04:12:25.417754 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.417724    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-workload-socket\") pod \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") "
Jul 10 04:12:25.417976 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.417757    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-podinfo\") pod \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") "
Jul 10 04:12:25.417976 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.417782    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-workload-certs\") pod \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") "
Jul 10 04:12:25.417976 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.417820    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-data\") pod \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") "
Jul 10 04:12:25.417976 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.417858    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-token\") pod \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\" (UID: \"1908ceaf-2cf1-4cff-a470-08e24ea13d09\") "
Jul 10 04:12:25.417976 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.417882    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1908ceaf-2cf1-4cff-a470-08e24ea13d09" (UID: "1908ceaf-2cf1-4cff-a470-08e24ea13d09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:12:25.418239 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.418121    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-credential-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:25.418239 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.418139    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1908ceaf-2cf1-4cff-a470-08e24ea13d09" (UID: "1908ceaf-2cf1-4cff-a470-08e24ea13d09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:12:25.418239 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.418153    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1908ceaf-2cf1-4cff-a470-08e24ea13d09" (UID: "1908ceaf-2cf1-4cff-a470-08e24ea13d09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:12:25.418239 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.418190    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1908ceaf-2cf1-4cff-a470-08e24ea13d09" (UID: "1908ceaf-2cf1-4cff-a470-08e24ea13d09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:12:25.418921 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.418887    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-data" (OuterVolumeSpecName: "istio-data") pod "1908ceaf-2cf1-4cff-a470-08e24ea13d09" (UID: "1908ceaf-2cf1-4cff-a470-08e24ea13d09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:12:25.420028 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.420001    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1908ceaf-2cf1-4cff-a470-08e24ea13d09-kube-api-access-p6xwm" (OuterVolumeSpecName: "kube-api-access-p6xwm") pod "1908ceaf-2cf1-4cff-a470-08e24ea13d09" (UID: "1908ceaf-2cf1-4cff-a470-08e24ea13d09"). InnerVolumeSpecName "kube-api-access-p6xwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:12:25.420133 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.420025    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1908ceaf-2cf1-4cff-a470-08e24ea13d09" (UID: "1908ceaf-2cf1-4cff-a470-08e24ea13d09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:12:25.420194 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.420176    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-token" (OuterVolumeSpecName: "istio-token") pod "1908ceaf-2cf1-4cff-a470-08e24ea13d09" (UID: "1908ceaf-2cf1-4cff-a470-08e24ea13d09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:12:25.420473 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.420452    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1908ceaf-2cf1-4cff-a470-08e24ea13d09" (UID: "1908ceaf-2cf1-4cff-a470-08e24ea13d09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 04:12:25.519292 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.519252    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-workload-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:25.519292 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.519286    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-podinfo\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:25.519292 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.519295    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-workload-certs\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:25.519292 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.519304    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-data\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:25.519566 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.519313    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-token\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:25.519566 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.519321    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6xwm\" (UniqueName: \"kubernetes.io/projected/1908ceaf-2cf1-4cff-a470-08e24ea13d09-kube-api-access-p6xwm\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:25.519566 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.519330    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istiod-ca-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:25.519566 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.519338    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1908ceaf-2cf1-4cff-a470-08e24ea13d09-istio-envoy\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:25.831821 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.831784    2561 generic.go:356] "Generic (PLEG): container finished" podID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerID="830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd" exitCode=0
Jul 10 04:12:25.831983 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.831850    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"
Jul 10 04:12:25.831983 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.831869    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" event={"ID":"1908ceaf-2cf1-4cff-a470-08e24ea13d09","Type":"ContainerDied","Data":"830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd"}
Jul 10 04:12:25.831983 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.831946    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm" event={"ID":"1908ceaf-2cf1-4cff-a470-08e24ea13d09","Type":"ContainerDied","Data":"ddbbfe9612d2b0bdee9b3ebd5df305c4f54cc392d33dd0f184002b9a39ff9c15"}
Jul 10 04:12:25.831983 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.831969    2561 scope.go:117] "RemoveContainer" containerID="830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd"
Jul 10 04:12:25.842614 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.842594    2561 scope.go:117] "RemoveContainer" containerID="dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10"
Jul 10 04:12:25.851157 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.851137    2561 scope.go:117] "RemoveContainer" containerID="830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd"
Jul 10 04:12:25.851400 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:12:25.851373    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd\": container with ID starting with 830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd not found: ID does not exist" containerID="830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd"
Jul 10 04:12:25.851512 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.851403    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd"} err="failed to get container status \"830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd\": rpc error: code = NotFound desc = could not find container \"830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd\": container with ID starting with 830e1c8eb9d06b356240a3b657f9ef6a14674233d7d136f536771036359de4fd not found: ID does not exist"
Jul 10 04:12:25.851512 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.851425    2561 scope.go:117] "RemoveContainer" containerID="dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10"
Jul 10 04:12:25.851760 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:12:25.851735    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10\": container with ID starting with dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10 not found: ID does not exist" containerID="dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10"
Jul 10 04:12:25.851819 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.851770    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10"} err="failed to get container status \"dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10\": rpc error: code = NotFound desc = could not find container \"dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10\": container with ID starting with dc8f5c4ebe36d705d1e9586bb5546d83e0d4ae00813b60b62170e64371789c10 not found: ID does not exist"
Jul 10 04:12:25.855068 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.855038    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"]
Jul 10 04:12:25.858558 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:25.858534    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f48e73-openshift-default-d848cc785-v7zhm"]
Jul 10 04:12:26.994759 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:26.994727    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" path="/var/lib/kubelet/pods/1908ceaf-2cf1-4cff-a470-08e24ea13d09/volumes"
Jul 10 04:12:30.760361 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:30.760323    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"]
Jul 10 04:12:30.760829 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:30.760570    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4" podUID="20380e00-a881-4e18-ad48-c58ddc336a08" containerName="maas-api" containerID="cri-o://5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234" gracePeriod=30
Jul 10 04:12:30.790569 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:30.790542    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:31.008142 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.008116    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:12:31.169888 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.169856    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pcss\" (UniqueName: \"kubernetes.io/projected/20380e00-a881-4e18-ad48-c58ddc336a08-kube-api-access-5pcss\") pod \"20380e00-a881-4e18-ad48-c58ddc336a08\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") "
Jul 10 04:12:31.169888 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.169896    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20380e00-a881-4e18-ad48-c58ddc336a08-openshift-service-ca\") pod \"20380e00-a881-4e18-ad48-c58ddc336a08\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") "
Jul 10 04:12:31.170099 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.169949    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20380e00-a881-4e18-ad48-c58ddc336a08-maas-api-tls\") pod \"20380e00-a881-4e18-ad48-c58ddc336a08\" (UID: \"20380e00-a881-4e18-ad48-c58ddc336a08\") "
Jul 10 04:12:31.170324 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.170298    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20380e00-a881-4e18-ad48-c58ddc336a08-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "20380e00-a881-4e18-ad48-c58ddc336a08" (UID: "20380e00-a881-4e18-ad48-c58ddc336a08"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:12:31.172078 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.172053    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20380e00-a881-4e18-ad48-c58ddc336a08-kube-api-access-5pcss" (OuterVolumeSpecName: "kube-api-access-5pcss") pod "20380e00-a881-4e18-ad48-c58ddc336a08" (UID: "20380e00-a881-4e18-ad48-c58ddc336a08"). InnerVolumeSpecName "kube-api-access-5pcss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:12:31.172181 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.172086    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20380e00-a881-4e18-ad48-c58ddc336a08-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "20380e00-a881-4e18-ad48-c58ddc336a08" (UID: "20380e00-a881-4e18-ad48-c58ddc336a08"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:12:31.270823 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.270788    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20380e00-a881-4e18-ad48-c58ddc336a08-maas-api-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:31.270823 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.270818    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pcss\" (UniqueName: \"kubernetes.io/projected/20380e00-a881-4e18-ad48-c58ddc336a08-kube-api-access-5pcss\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:31.270823 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.270828    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20380e00-a881-4e18-ad48-c58ddc336a08-openshift-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:12:31.869149 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.869115    2561 generic.go:356] "Generic (PLEG): container finished" podID="20380e00-a881-4e18-ad48-c58ddc336a08" containerID="5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234" exitCode=0
Jul 10 04:12:31.869563 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.869176    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"
Jul 10 04:12:31.869563 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.869187    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4" event={"ID":"20380e00-a881-4e18-ad48-c58ddc336a08","Type":"ContainerDied","Data":"5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234"}
Jul 10 04:12:31.869563 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.869254    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4" event={"ID":"20380e00-a881-4e18-ad48-c58ddc336a08","Type":"ContainerDied","Data":"acdba1fbf399250114312db145647868207bb5f0dd26250281a5dd02fb75c72b"}
Jul 10 04:12:31.869563 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.869276    2561 scope.go:117] "RemoveContainer" containerID="5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234"
Jul 10 04:12:31.879856 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.879833    2561 scope.go:117] "RemoveContainer" containerID="5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234"
Jul 10 04:12:31.880126 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:12:31.880106    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234\": container with ID starting with 5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234 not found: ID does not exist" containerID="5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234"
Jul 10 04:12:31.880177 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.880134    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234"} err="failed to get container status \"5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234\": rpc error: code = NotFound desc = could not find container \"5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234\": container with ID starting with 5d69a9789601e4a87b9c36fd914ec3cf8bb1270d90f59c5b620911930c69c234 not found: ID does not exist"
Jul 10 04:12:31.891514 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.891375    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"]
Jul 10 04:12:31.893948 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:31.893928    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a335e2-7cbc856894-2mcd4"]
Jul 10 04:12:32.994084 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:32.994050    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20380e00-a881-4e18-ad48-c58ddc336a08" path="/var/lib/kubelet/pods/20380e00-a881-4e18-ad48-c58ddc336a08/volumes"
Jul 10 04:12:35.482139 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:35.482109    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:41.248058 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:41.247947    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:41.306254 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:41.306213    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:46.212558 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:46.212525    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:46.464938 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:46.464850    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:46.887456 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:46.887422    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:47.241843 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.241770    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"]
Jul 10 04:12:47.242211 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.242145    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20380e00-a881-4e18-ad48-c58ddc336a08" containerName="maas-api"
Jul 10 04:12:47.242211 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.242157    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="20380e00-a881-4e18-ad48-c58ddc336a08" containerName="maas-api"
Jul 10 04:12:47.242211 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.242173    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerName="istio-proxy"
Jul 10 04:12:47.242211 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.242178    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerName="istio-proxy"
Jul 10 04:12:47.242211 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.242186    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29978b1d-4bb8-48dc-8056-1ada76651e6e" containerName="revoke-keys"
Jul 10 04:12:47.242211 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.242192    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="29978b1d-4bb8-48dc-8056-1ada76651e6e" containerName="revoke-keys"
Jul 10 04:12:47.242211 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.242208    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerName="istio-proxy"
Jul 10 04:12:47.242211 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.242214    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerName="istio-proxy"
Jul 10 04:12:47.242491 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.242282    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="29978b1d-4bb8-48dc-8056-1ada76651e6e" containerName="revoke-keys"
Jul 10 04:12:47.242491 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.242292    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="20380e00-a881-4e18-ad48-c58ddc336a08" containerName="maas-api"
Jul 10 04:12:47.242491 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.242300    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerName="istio-proxy"
Jul 10 04:12:47.242491 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.242307    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1908ceaf-2cf1-4cff-a470-08e24ea13d09" containerName="istio-proxy"
Jul 10 04:12:47.246777 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.246757    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.249035 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.249009    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 04:12:47.249148 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.249074    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-937cca-openshift-default-dockercfg-x5s8v\""
Jul 10 04:12:47.257816 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.257795    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"]
Jul 10 04:12:47.259429 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.259385    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:47.300024 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.299977    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-workload-socket\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.300206 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.300034    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-credential-socket\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.300206 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.300080    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdd4x\" (UniqueName: \"kubernetes.io/projected/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-kube-api-access-pdd4x\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.300206 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.300111    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-workload-certs\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.300206 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.300150    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istiod-ca-cert\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.300206 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.300175    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-data\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.300206 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.300202    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-token\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.300414 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.300227    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-envoy\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.300414 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.300276    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-podinfo\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.398030 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.397996    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:47.401191 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401164    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-workload-socket\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.401321 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401206    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-credential-socket\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.401321 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401238    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdd4x\" (UniqueName: \"kubernetes.io/projected/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-kube-api-access-pdd4x\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.401321 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401270    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-workload-certs\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.401321 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401311    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istiod-ca-cert\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.401539 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401338    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-data\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.401539 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401368    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-token\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.401539 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401396    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-envoy\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.401539 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401427    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-podinfo\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.401765 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401620    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-workload-socket\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.401765 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401640    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-workload-certs\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.401873 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401772    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-data\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.401873 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.401802    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-credential-socket\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.402069 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.402046    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istiod-ca-cert\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.403689 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.403638    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-envoy\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.404053 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.404032    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-podinfo\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.408822 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.408802    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-token\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.409346 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.409323    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdd4x\" (UniqueName: \"kubernetes.io/projected/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-kube-api-access-pdd4x\") pod \"e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") " pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.557257 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.557163    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:47.695394 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.695365    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"]
Jul 10 04:12:47.697432 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:12:47.697393    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d513a68_dbe4_4c28_b515_8dd45f8b26d4.slice/crio-6b7099e9eed9919e657b38b9a8fa435a54518ee6004ba0ef412c2f077bd79698 WatchSource:0}: Error finding container 6b7099e9eed9919e657b38b9a8fa435a54518ee6004ba0ef412c2f077bd79698: Status 404 returned error can't find the container with id 6b7099e9eed9919e657b38b9a8fa435a54518ee6004ba0ef412c2f077bd79698
Jul 10 04:12:47.699516 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.699479    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:12:47.699591 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.699572    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:12:47.699636 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.699621    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:12:47.965767 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.965730    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" event={"ID":"7d513a68-dbe4-4c28-b515-8dd45f8b26d4","Type":"ContainerStarted","Data":"96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a"}
Jul 10 04:12:47.965945 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.965787    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" event={"ID":"7d513a68-dbe4-4c28-b515-8dd45f8b26d4","Type":"ContainerStarted","Data":"6b7099e9eed9919e657b38b9a8fa435a54518ee6004ba0ef412c2f077bd79698"}
Jul 10 04:12:47.987170 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:47.987122    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" podStartSLOduration=0.987105457 podStartE2EDuration="987.105457ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:12:47.986213042 +0000 UTC m=+2429.477208916" watchObservedRunningTime="2026-07-10 04:12:47.987105457 +0000 UTC m=+2429.478101331"
Jul 10 04:12:48.557632 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:48.557587    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:48.562535 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:48.562508    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:48.971354 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:48.971320    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:48.972285 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:48.972266    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:12:51.527268 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.527235    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:51.625105 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.625074    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:51.735389 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.735355    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"]
Jul 10 04:12:51.739083 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.739062    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:51.743169 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.743147    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 04:12:51.743405 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.743385    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 04:12:51.744107 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.744089    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-937cca-serving-cert\""
Jul 10 04:12:51.744107 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.744103    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6g9gw\""
Jul 10 04:12:51.760142 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.760117    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"]
Jul 10 04:12:51.785174 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.785107    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:51.838189 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.838146    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a55577d-8f8c-4ef7-bc32-e04840d08f09-openshift-service-ca\") pod \"maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:51.838384 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.838201    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vzs7\" (UniqueName: \"kubernetes.io/projected/6a55577d-8f8c-4ef7-bc32-e04840d08f09-kube-api-access-6vzs7\") pod \"maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:51.838384 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.838219    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a55577d-8f8c-4ef7-bc32-e04840d08f09-maas-api-tls\") pod \"maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:51.939365 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.939329    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a55577d-8f8c-4ef7-bc32-e04840d08f09-openshift-service-ca\") pod \"maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:51.939536 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.939377    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vzs7\" (UniqueName: \"kubernetes.io/projected/6a55577d-8f8c-4ef7-bc32-e04840d08f09-kube-api-access-6vzs7\") pod \"maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:51.939536 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.939393    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a55577d-8f8c-4ef7-bc32-e04840d08f09-maas-api-tls\") pod \"maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:51.939536 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:12:51.939513    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-serving-cert: secret "maas-api-e2e-shared-a-937cca-serving-cert" not found
Jul 10 04:12:51.939677 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:12:51.939576    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a55577d-8f8c-4ef7-bc32-e04840d08f09-maas-api-tls podName:6a55577d-8f8c-4ef7-bc32-e04840d08f09 nodeName:}" failed. No retries permitted until 2026-07-10 04:12:52.439559706 +0000 UTC m=+2433.930555557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6a55577d-8f8c-4ef7-bc32-e04840d08f09-maas-api-tls") pod "maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh" (UID: "6a55577d-8f8c-4ef7-bc32-e04840d08f09") : secret "maas-api-e2e-shared-a-937cca-serving-cert" not found
Jul 10 04:12:51.940125 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.940106    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a55577d-8f8c-4ef7-bc32-e04840d08f09-openshift-service-ca\") pod \"maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:51.952209 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:51.952181    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vzs7\" (UniqueName: \"kubernetes.io/projected/6a55577d-8f8c-4ef7-bc32-e04840d08f09-kube-api-access-6vzs7\") pod \"maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:52.002616 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:52.002568    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:52.247297 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:52.247269    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:52.348011 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:52.347979    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:52.442442 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:52.442406    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a55577d-8f8c-4ef7-bc32-e04840d08f09-maas-api-tls\") pod \"maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:52.444939 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:52.444914    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a55577d-8f8c-4ef7-bc32-e04840d08f09-maas-api-tls\") pod \"maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:52.650459 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:52.650426    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:52.806764 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:52.806727    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"]
Jul 10 04:12:52.808283 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:12:52.808253    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a55577d_8f8c_4ef7_bc32_e04840d08f09.slice/crio-52a09979b921d47a8a12f5a29fc42e5a9b3062cba48078ed0cff125f589c11aa WatchSource:0}: Error finding container 52a09979b921d47a8a12f5a29fc42e5a9b3062cba48078ed0cff125f589c11aa: Status 404 returned error can't find the container with id 52a09979b921d47a8a12f5a29fc42e5a9b3062cba48078ed0cff125f589c11aa
Jul 10 04:12:52.998034 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:52.997945    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh" event={"ID":"6a55577d-8f8c-4ef7-bc32-e04840d08f09","Type":"ContainerStarted","Data":"52a09979b921d47a8a12f5a29fc42e5a9b3062cba48078ed0cff125f589c11aa"}
Jul 10 04:12:54.005446 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:54.005407    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh" event={"ID":"6a55577d-8f8c-4ef7-bc32-e04840d08f09","Type":"ContainerStarted","Data":"52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661"}
Jul 10 04:12:54.005923 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:54.005479    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:12:54.025593 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:54.025543    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh" podStartSLOduration=2.560291535 podStartE2EDuration="3.025528008s" totalImagesPullingTime="465.236473ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:12:54.023822939 +0000 UTC m=+2435.514818825" watchObservedRunningTime="2026-07-10 04:12:54.025528008 +0000 UTC m=+2435.516523880"
Jul 10 04:12:56.299032 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:56.298998    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:56.690808 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:56.690768    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:57.025866 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:57.025788    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:57.152151 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:57.152113    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:58.621789 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:58.621738    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:12:58.721915 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:12:58.721885    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:00.017336 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.017307    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:13:00.648361 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.648324    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:00.728086 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.728048    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"]
Jul 10 04:13:00.731960 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.731928    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:00.735028 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.735004    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-3505cb-serving-cert\""
Jul 10 04:13:00.752364 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.752329    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"]
Jul 10 04:13:00.769459 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.769413    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:00.829284 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.829248    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdj2c\" (UniqueName: \"kubernetes.io/projected/98fec6aa-0cf6-4202-a074-417e0b6f8665-kube-api-access-rdj2c\") pod \"maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d\" (UID: \"98fec6aa-0cf6-4202-a074-417e0b6f8665\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:00.829495 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.829301    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98fec6aa-0cf6-4202-a074-417e0b6f8665-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d\" (UID: \"98fec6aa-0cf6-4202-a074-417e0b6f8665\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:00.829495 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.829356    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98fec6aa-0cf6-4202-a074-417e0b6f8665-maas-api-tls\") pod \"maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d\" (UID: \"98fec6aa-0cf6-4202-a074-417e0b6f8665\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:00.856381 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.856345    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:00.930179 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.930081    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdj2c\" (UniqueName: \"kubernetes.io/projected/98fec6aa-0cf6-4202-a074-417e0b6f8665-kube-api-access-rdj2c\") pod \"maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d\" (UID: \"98fec6aa-0cf6-4202-a074-417e0b6f8665\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:00.930179 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.930137    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98fec6aa-0cf6-4202-a074-417e0b6f8665-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d\" (UID: \"98fec6aa-0cf6-4202-a074-417e0b6f8665\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:00.930453 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.930180    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98fec6aa-0cf6-4202-a074-417e0b6f8665-maas-api-tls\") pod \"maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d\" (UID: \"98fec6aa-0cf6-4202-a074-417e0b6f8665\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:00.931003 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.930974    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98fec6aa-0cf6-4202-a074-417e0b6f8665-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d\" (UID: \"98fec6aa-0cf6-4202-a074-417e0b6f8665\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:00.933000 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.932965    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98fec6aa-0cf6-4202-a074-417e0b6f8665-maas-api-tls\") pod \"maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d\" (UID: \"98fec6aa-0cf6-4202-a074-417e0b6f8665\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:00.938283 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.938254    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdj2c\" (UniqueName: \"kubernetes.io/projected/98fec6aa-0cf6-4202-a074-417e0b6f8665-kube-api-access-rdj2c\") pod \"maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d\" (UID: \"98fec6aa-0cf6-4202-a074-417e0b6f8665\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:00.953711 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:00.953683    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:01.046389 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:01.046322    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:01.202595 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:01.202566    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"]
Jul 10 04:13:01.203838 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:13:01.203795    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98fec6aa_0cf6_4202_a074_417e0b6f8665.slice/crio-f4d0107229d1ad925720988ac1e2d07d9e99734f972a9d756bd94450ebd726a7 WatchSource:0}: Error finding container f4d0107229d1ad925720988ac1e2d07d9e99734f972a9d756bd94450ebd726a7: Status 404 returned error can't find the container with id f4d0107229d1ad925720988ac1e2d07d9e99734f972a9d756bd94450ebd726a7
Jul 10 04:13:01.300203 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:01.300155    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:01.401201 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:01.401167    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:02.059406 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:02.059315    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d" event={"ID":"98fec6aa-0cf6-4202-a074-417e0b6f8665","Type":"ContainerStarted","Data":"c51d40940a849ee4f06a2ba342a505a86db611da2c2734ebc1c552441e483183"}
Jul 10 04:13:02.059406 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:02.059369    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d" event={"ID":"98fec6aa-0cf6-4202-a074-417e0b6f8665","Type":"ContainerStarted","Data":"f4d0107229d1ad925720988ac1e2d07d9e99734f972a9d756bd94450ebd726a7"}
Jul 10 04:13:02.059406 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:02.059396    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:02.074605 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:02.074551    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d" podStartSLOduration=1.7069966060000001 podStartE2EDuration="2.07453114s" totalImagesPullingTime="367.534534ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:13:02.073797438 +0000 UTC m=+2443.564793314" watchObservedRunningTime="2026-07-10 04:13:02.07453114 +0000 UTC m=+2443.565527013"
Jul 10 04:13:03.066948 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:03.066862    2561 generic.go:356] "Generic (PLEG): container finished" podID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerID="96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a" exitCode=0
Jul 10 04:13:03.066948 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:03.066934    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" event={"ID":"7d513a68-dbe4-4c28-b515-8dd45f8b26d4","Type":"ContainerDied","Data":"96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a"}
Jul 10 04:13:03.067452 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:03.067325    2561 scope.go:117] "RemoveContainer" containerID="96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a"
Jul 10 04:13:03.068359 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:03.068333    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:13:03.068428 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:03.068414    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:13:03.068465 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:03.068450    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 04:13:03.971431 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:03.971390    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:13:04.074582 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:04.074546    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" event={"ID":"7d513a68-dbe4-4c28-b515-8dd45f8b26d4","Type":"ContainerStarted","Data":"43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7"}
Jul 10 04:13:04.557996 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:04.557957    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:13:04.559283 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:04.559257    2561 patch_prober.go:28] interesting pod/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Jul 10 04:13:04.559405 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:04.559306    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Jul 10 04:13:05.430663 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:05.430625    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:05.558360 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:05.558329    2561 patch_prober.go:28] interesting pod/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Jul 10 04:13:05.558543 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:05.558383    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Jul 10 04:13:06.557532 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:06.557492    2561 patch_prober.go:28] interesting pod/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Jul 10 04:13:06.557967 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:06.557548    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Jul 10 04:13:07.558312 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:07.558274    2561 patch_prober.go:28] interesting pod/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Jul 10 04:13:07.558712 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:07.558333    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Jul 10 04:13:08.073075 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:08.073044    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:13:08.561311 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:08.561224    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:13:08.561743 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:08.561715    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:13:08.562344 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:08.562328    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:13:13.710936 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:13.710896    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:13.812452 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:13.812422    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:16.438633 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:16.438597    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:22.004630 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:22.004590    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:22.105403 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:22.105369    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:13:49.667171 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:13:49.667129    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:14:18.185865 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:14:18.185787    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:15:00.133235 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.133199    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh"]
Jul 10 04:15:00.136367 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.136350    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3505cb-29727615-jp4pt"]
Jul 10 04:15:00.136482 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.136461    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh"
Jul 10 04:15:00.138810 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.138791    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:15:00.139578 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.139562    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727615-t4jpr"]
Jul 10 04:15:00.139686 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.139644    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3505cb-29727615-jp4pt"
Jul 10 04:15:00.143438 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.143418    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727615-t4jpr"
Jul 10 04:15:00.143553 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.143434    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh"]
Jul 10 04:15:00.147053 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.147011    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3505cb-29727615-jp4pt"]
Jul 10 04:15:00.151602 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.151580    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh"
Jul 10 04:15:00.167590 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.167561    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727615-t4jpr"]
Jul 10 04:15:00.189282 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.189255    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3505cb-29727615-jp4pt"
Jul 10 04:15:00.197129 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.197107    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727615-t4jpr"
Jul 10 04:15:00.325432 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.325174    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh"]
Jul 10 04:15:00.326931 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:15:00.326899    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeff4f42_fba8_4e06_b469_565eec2cdd1a.slice/crio-d808c30295512333b27656d89e9b5b5a082ff9ef7b8729db72b70ef0c1f818eb WatchSource:0}: Error finding container d808c30295512333b27656d89e9b5b5a082ff9ef7b8729db72b70ef0c1f818eb: Status 404 returned error can't find the container with id d808c30295512333b27656d89e9b5b5a082ff9ef7b8729db72b70ef0c1f818eb
Jul 10 04:15:00.356816 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.356787    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3505cb-29727615-jp4pt"]
Jul 10 04:15:00.359198 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:15:00.359166    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308028de_939b_4e66_ab72_fb9261cce6b0.slice/crio-9c91343eedc05d5de9ce00c7f781a30145f0c8078544ea3898cff291deccded4 WatchSource:0}: Error finding container 9c91343eedc05d5de9ce00c7f781a30145f0c8078544ea3898cff291deccded4: Status 404 returned error can't find the container with id 9c91343eedc05d5de9ce00c7f781a30145f0c8078544ea3898cff291deccded4
Jul 10 04:15:00.380021 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.379997    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727615-t4jpr"]
Jul 10 04:15:00.382227 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:15:00.382185    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded760aa2_906e_4fa6_87f8_11b09ef5e807.slice/crio-392043e122fe1718b43ee31d3c1e811ffb0c43d5187d4b3f9fb95104d9ec4884 WatchSource:0}: Error finding container 392043e122fe1718b43ee31d3c1e811ffb0c43d5187d4b3f9fb95104d9ec4884: Status 404 returned error can't find the container with id 392043e122fe1718b43ee31d3c1e811ffb0c43d5187d4b3f9fb95104d9ec4884
Jul 10 04:15:00.769926 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.769890    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh" event={"ID":"aeff4f42-fba8-4e06-b469-565eec2cdd1a","Type":"ContainerStarted","Data":"f6442e6e235158e49d3b215f6058db7cf75090512e217d802157cf2cad14930c"}
Jul 10 04:15:00.770097 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.769952    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh" event={"ID":"aeff4f42-fba8-4e06-b469-565eec2cdd1a","Type":"ContainerStarted","Data":"d808c30295512333b27656d89e9b5b5a082ff9ef7b8729db72b70ef0c1f818eb"}
Jul 10 04:15:00.771215 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.771182    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3505cb-29727615-jp4pt" event={"ID":"308028de-939b-4e66-ab72-fb9261cce6b0","Type":"ContainerStarted","Data":"9c91343eedc05d5de9ce00c7f781a30145f0c8078544ea3898cff291deccded4"}
Jul 10 04:15:00.772509 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.772485    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727615-t4jpr" event={"ID":"ed760aa2-906e-4fa6-87f8-11b09ef5e807","Type":"ContainerStarted","Data":"692afed7ee7871c34f00b9104fc656487a094d3f775fb776b893d879a92d4de3"}
Jul 10 04:15:00.772633 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.772537    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727615-t4jpr" event={"ID":"ed760aa2-906e-4fa6-87f8-11b09ef5e807","Type":"ContainerStarted","Data":"392043e122fe1718b43ee31d3c1e811ffb0c43d5187d4b3f9fb95104d9ec4884"}
Jul 10 04:15:00.786850 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.786803    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh" podStartSLOduration=0.786789904 podStartE2EDuration="786.789904ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:15:00.785142483 +0000 UTC m=+2562.276138357" watchObservedRunningTime="2026-07-10 04:15:00.786789904 +0000 UTC m=+2562.277785777"
Jul 10 04:15:00.800374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:00.800335    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727615-t4jpr" podStartSLOduration=0.800323204 podStartE2EDuration="800.323204ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:15:00.798205416 +0000 UTC m=+2562.289201291" watchObservedRunningTime="2026-07-10 04:15:00.800323204 +0000 UTC m=+2562.291319075"
Jul 10 04:15:01.780749 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:01.780643    2561 generic.go:356] "Generic (PLEG): container finished" podID="ed760aa2-906e-4fa6-87f8-11b09ef5e807" containerID="692afed7ee7871c34f00b9104fc656487a094d3f775fb776b893d879a92d4de3" exitCode=0
Jul 10 04:15:01.781138 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:01.780778    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727615-t4jpr" event={"ID":"ed760aa2-906e-4fa6-87f8-11b09ef5e807","Type":"ContainerDied","Data":"692afed7ee7871c34f00b9104fc656487a094d3f775fb776b893d879a92d4de3"}
Jul 10 04:15:01.782310 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:01.782285    2561 generic.go:356] "Generic (PLEG): container finished" podID="aeff4f42-fba8-4e06-b469-565eec2cdd1a" containerID="f6442e6e235158e49d3b215f6058db7cf75090512e217d802157cf2cad14930c" exitCode=0
Jul 10 04:15:01.782433 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:01.782365    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh" event={"ID":"aeff4f42-fba8-4e06-b469-565eec2cdd1a","Type":"ContainerDied","Data":"f6442e6e235158e49d3b215f6058db7cf75090512e217d802157cf2cad14930c"}
Jul 10 04:15:01.783845 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:01.783819    2561 generic.go:356] "Generic (PLEG): container finished" podID="308028de-939b-4e66-ab72-fb9261cce6b0" containerID="406635310b35eda5720bd2dc38021840b7d7cd2793a3c7c5eceb491fb33d8414" exitCode=0
Jul 10 04:15:01.783949 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:01.783871    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3505cb-29727615-jp4pt" event={"ID":"308028de-939b-4e66-ab72-fb9261cce6b0","Type":"ContainerDied","Data":"406635310b35eda5720bd2dc38021840b7d7cd2793a3c7c5eceb491fb33d8414"}
Jul 10 04:15:02.943903 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:02.943876    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727615-t4jpr"
Jul 10 04:15:02.956781 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:02.956758    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3505cb-29727615-jp4pt"
Jul 10 04:15:02.998159 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:02.998133    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh"
Jul 10 04:15:03.797596 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:03.797558    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727615-t4jpr"
Jul 10 04:15:03.797596 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:03.797577    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727615-t4jpr" event={"ID":"ed760aa2-906e-4fa6-87f8-11b09ef5e807","Type":"ContainerDied","Data":"392043e122fe1718b43ee31d3c1e811ffb0c43d5187d4b3f9fb95104d9ec4884"}
Jul 10 04:15:03.797963 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:03.797680    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="392043e122fe1718b43ee31d3c1e811ffb0c43d5187d4b3f9fb95104d9ec4884"
Jul 10 04:15:03.799041 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:03.799012    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh"
Jul 10 04:15:03.799167 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:03.799067    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh" event={"ID":"aeff4f42-fba8-4e06-b469-565eec2cdd1a","Type":"ContainerDied","Data":"d808c30295512333b27656d89e9b5b5a082ff9ef7b8729db72b70ef0c1f818eb"}
Jul 10 04:15:03.799167 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:03.799124    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d808c30295512333b27656d89e9b5b5a082ff9ef7b8729db72b70ef0c1f818eb"
Jul 10 04:15:03.800711 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:03.800689    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3505cb-29727615-jp4pt" event={"ID":"308028de-939b-4e66-ab72-fb9261cce6b0","Type":"ContainerDied","Data":"9c91343eedc05d5de9ce00c7f781a30145f0c8078544ea3898cff291deccded4"}
Jul 10 04:15:03.800711 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:03.800704    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3505cb-29727615-jp4pt"
Jul 10 04:15:03.800846 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:03.800732    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c91343eedc05d5de9ce00c7f781a30145f0c8078544ea3898cff291deccded4"
Jul 10 04:15:22.590780 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:22.590733    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:15:23.749826 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:23.749790    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:15:56.297978 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:56.297898    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:15:57.274314 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:15:57.274284    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:16:07.921999 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:16:07.921964    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:16:44.408987 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:16:44.408934    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:17:19.253761 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:17:19.253681    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 04:17:19.273150 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:17:19.273126    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 04:17:34.222628 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:17:34.222571    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:17:35.207705 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:17:35.207666    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:17:46.384057 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:17:46.384023    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:17:59.314816 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:17:59.314772    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:17:59.412488 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:17:59.412447    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:18:03.704903 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:18:03.704846    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:18:03.805153 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:18:03.805120    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:18:13.381022 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:18:13.380983    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:18:14.435547 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:18:14.435518    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:18:14.535548 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:18:14.535514    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:18:22.076854 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:18:22.076818    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:18:22.689245 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:18:22.689210    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:18:22.788214 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:18:22.788176    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:07.119779 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:07.119744    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:08.072185 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:08.072154    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:09.064605 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.064571    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7369-139861127701296"]
Jul 10 04:19:09.064996 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.064951    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="308028de-939b-4e66-ab72-fb9261cce6b0" containerName="cleanup"
Jul 10 04:19:09.064996 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.064963    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="308028de-939b-4e66-ab72-fb9261cce6b0" containerName="cleanup"
Jul 10 04:19:09.064996 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.064984    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeff4f42-fba8-4e06-b469-565eec2cdd1a" containerName="cleanup"
Jul 10 04:19:09.064996 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.064990    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeff4f42-fba8-4e06-b469-565eec2cdd1a" containerName="cleanup"
Jul 10 04:19:09.065125 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.065001    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed760aa2-906e-4fa6-87f8-11b09ef5e807" containerName="cleanup"
Jul 10 04:19:09.065125 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.065006    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed760aa2-906e-4fa6-87f8-11b09ef5e807" containerName="cleanup"
Jul 10 04:19:09.065125 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.065071    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeff4f42-fba8-4e06-b469-565eec2cdd1a" containerName="cleanup"
Jul 10 04:19:09.065125 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.065079    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="308028de-939b-4e66-ab72-fb9261cce6b0" containerName="cleanup"
Jul 10 04:19:09.065125 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.065088    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed760aa2-906e-4fa6-87f8-11b09ef5e807" containerName="cleanup"
Jul 10 04:19:09.068085 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.068068    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861127701296"
Jul 10 04:19:09.070526 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.070502    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-5xvrj\""
Jul 10 04:19:09.073400 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.073372    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7369-139861127701296"]
Jul 10 04:19:09.168910 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.168873    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vbp8\" (UniqueName: \"kubernetes.io/projected/01cbe5fb-5bbf-4085-82a4-1272f7096f49-kube-api-access-7vbp8\") pod \"test-curl-7369-139861127701296\" (UID: \"01cbe5fb-5bbf-4085-82a4-1272f7096f49\") " pod="openshift-ingress/test-curl-7369-139861127701296"
Jul 10 04:19:09.270143 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.270088    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vbp8\" (UniqueName: \"kubernetes.io/projected/01cbe5fb-5bbf-4085-82a4-1272f7096f49-kube-api-access-7vbp8\") pod \"test-curl-7369-139861127701296\" (UID: \"01cbe5fb-5bbf-4085-82a4-1272f7096f49\") " pod="openshift-ingress/test-curl-7369-139861127701296"
Jul 10 04:19:09.277627 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.277608    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vbp8\" (UniqueName: \"kubernetes.io/projected/01cbe5fb-5bbf-4085-82a4-1272f7096f49-kube-api-access-7vbp8\") pod \"test-curl-7369-139861127701296\" (UID: \"01cbe5fb-5bbf-4085-82a4-1272f7096f49\") " pod="openshift-ingress/test-curl-7369-139861127701296"
Jul 10 04:19:09.377787 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.377751    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861127701296"
Jul 10 04:19:09.515116 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.515093    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7369-139861127701296"]
Jul 10 04:19:09.516630 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:19:09.516606    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01cbe5fb_5bbf_4085_82a4_1272f7096f49.slice/crio-42a8a76973bfad1e66c24056a3212d6802c3d5660d65755f63249f3b36003744 WatchSource:0}: Error finding container 42a8a76973bfad1e66c24056a3212d6802c3d5660d65755f63249f3b36003744: Status 404 returned error can't find the container with id 42a8a76973bfad1e66c24056a3212d6802c3d5660d65755f63249f3b36003744
Jul 10 04:19:09.517969 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:09.517948    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 04:19:10.312298 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:10.312268    2561 generic.go:356] "Generic (PLEG): container finished" podID="01cbe5fb-5bbf-4085-82a4-1272f7096f49" containerID="b6a19ce5a807a56d95b753e51d7be06a045c28f5a6bdb4d28058f47ab8f5bece" exitCode=0
Jul 10 04:19:10.312638 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:10.312351    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7369-139861127701296" event={"ID":"01cbe5fb-5bbf-4085-82a4-1272f7096f49","Type":"ContainerDied","Data":"b6a19ce5a807a56d95b753e51d7be06a045c28f5a6bdb4d28058f47ab8f5bece"}
Jul 10 04:19:10.312638 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:10.312395    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7369-139861127701296" event={"ID":"01cbe5fb-5bbf-4085-82a4-1272f7096f49","Type":"ContainerStarted","Data":"42a8a76973bfad1e66c24056a3212d6802c3d5660d65755f63249f3b36003744"}
Jul 10 04:19:11.459116 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.459087    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861127701296"
Jul 10 04:19:11.476829 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.476803    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7369-139861127701296_01cbe5fb-5bbf-4085-82a4-1272f7096f49/test-curl-7369-139861127701296/0.log"
Jul 10 04:19:11.484009 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.483986    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vbp8\" (UniqueName: \"kubernetes.io/projected/01cbe5fb-5bbf-4085-82a4-1272f7096f49-kube-api-access-7vbp8\") pod \"01cbe5fb-5bbf-4085-82a4-1272f7096f49\" (UID: \"01cbe5fb-5bbf-4085-82a4-1272f7096f49\") "
Jul 10 04:19:11.486096 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.486069    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01cbe5fb-5bbf-4085-82a4-1272f7096f49-kube-api-access-7vbp8" (OuterVolumeSpecName: "kube-api-access-7vbp8") pod "01cbe5fb-5bbf-4085-82a4-1272f7096f49" (UID: "01cbe5fb-5bbf-4085-82a4-1272f7096f49"). InnerVolumeSpecName "kube-api-access-7vbp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:19:11.501495 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.501434    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7369-139861127701296"]
Jul 10 04:19:11.507859 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.507824    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7369-139861127701296"]
Jul 10 04:19:11.584822 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.584787    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vbp8\" (UniqueName: \"kubernetes.io/projected/01cbe5fb-5bbf-4085-82a4-1272f7096f49-kube-api-access-7vbp8\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:11.632359 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.632326    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7369-139861127677616"]
Jul 10 04:19:11.632769 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.632753    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01cbe5fb-5bbf-4085-82a4-1272f7096f49" containerName="test-curl-7369-139861127701296"
Jul 10 04:19:11.632825 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.632770    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="01cbe5fb-5bbf-4085-82a4-1272f7096f49" containerName="test-curl-7369-139861127701296"
Jul 10 04:19:11.632865 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.632850    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="01cbe5fb-5bbf-4085-82a4-1272f7096f49" containerName="test-curl-7369-139861127701296"
Jul 10 04:19:11.635992 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.635973    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861127677616"
Jul 10 04:19:11.642087 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.642060    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7369-139861127677616"]
Jul 10 04:19:11.685539 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.685508    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t5wt\" (UniqueName: \"kubernetes.io/projected/5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b-kube-api-access-2t5wt\") pod \"test-curl-7369-139861127677616\" (UID: \"5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b\") " pod="openshift-ingress/test-curl-7369-139861127677616"
Jul 10 04:19:11.786575 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.786538    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t5wt\" (UniqueName: \"kubernetes.io/projected/5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b-kube-api-access-2t5wt\") pod \"test-curl-7369-139861127677616\" (UID: \"5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b\") " pod="openshift-ingress/test-curl-7369-139861127677616"
Jul 10 04:19:11.798200 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.798176    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t5wt\" (UniqueName: \"kubernetes.io/projected/5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b-kube-api-access-2t5wt\") pod \"test-curl-7369-139861127677616\" (UID: \"5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b\") " pod="openshift-ingress/test-curl-7369-139861127677616"
Jul 10 04:19:11.945931 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:11.945840    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861127677616"
Jul 10 04:19:12.080964 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:12.080942    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7369-139861127677616"]
Jul 10 04:19:12.082137 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:19:12.082112    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ab33a3f_eeee_4ee1_b85b_5c3a17ec944b.slice/crio-0bfeca9d953c0cbd30a07d1873e55c72773cb79da03358c41d544ecb92c13fae WatchSource:0}: Error finding container 0bfeca9d953c0cbd30a07d1873e55c72773cb79da03358c41d544ecb92c13fae: Status 404 returned error can't find the container with id 0bfeca9d953c0cbd30a07d1873e55c72773cb79da03358c41d544ecb92c13fae
Jul 10 04:19:12.325897 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:12.325859    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7369-139861127677616" event={"ID":"5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b","Type":"ContainerStarted","Data":"0bfeca9d953c0cbd30a07d1873e55c72773cb79da03358c41d544ecb92c13fae"}
Jul 10 04:19:12.327044 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:12.327029    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="42a8a76973bfad1e66c24056a3212d6802c3d5660d65755f63249f3b36003744"
Jul 10 04:19:12.327123 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:12.327058    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861127701296"
Jul 10 04:19:12.337338 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:19:12.337312    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7369-139861127701296\" is forbidden: User \"system:node:ip-10-0-140-167.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-167.ec2.internal' and this object" podUID="01cbe5fb-5bbf-4085-82a4-1272f7096f49" pod="openshift-ingress/test-curl-7369-139861127701296"
Jul 10 04:19:12.994514 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:12.994484    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01cbe5fb-5bbf-4085-82a4-1272f7096f49" path="/var/lib/kubelet/pods/01cbe5fb-5bbf-4085-82a4-1272f7096f49/volumes"
Jul 10 04:19:13.334309 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:13.334279    2561 generic.go:356] "Generic (PLEG): container finished" podID="5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b" containerID="a4bc219d710045bd08095f4b71b6dd3d027c3b158de4f6b7a66f36be173b1939" exitCode=0
Jul 10 04:19:13.334451 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:13.334351    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7369-139861127677616" event={"ID":"5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b","Type":"ContainerDied","Data":"a4bc219d710045bd08095f4b71b6dd3d027c3b158de4f6b7a66f36be173b1939"}
Jul 10 04:19:14.486328 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.486300    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861127677616"
Jul 10 04:19:14.506297 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.506265    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7369-139861127677616_5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b/test-curl-7369-139861127677616/0.log"
Jul 10 04:19:14.506488 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.506471    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t5wt\" (UniqueName: \"kubernetes.io/projected/5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b-kube-api-access-2t5wt\") pod \"5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b\" (UID: \"5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b\") "
Jul 10 04:19:14.508630 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.508598    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b-kube-api-access-2t5wt" (OuterVolumeSpecName: "kube-api-access-2t5wt") pod "5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b" (UID: "5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b"). InnerVolumeSpecName "kube-api-access-2t5wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:19:14.534558 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.534525    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7369-139861127677616"]
Jul 10 04:19:14.539756 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.539730    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7369-139861127677616"]
Jul 10 04:19:14.607501 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.607413    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t5wt\" (UniqueName: \"kubernetes.io/projected/5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b-kube-api-access-2t5wt\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:14.712029 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.711992    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7369-139861129640816"]
Jul 10 04:19:14.712376 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.712363    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b" containerName="test-curl-7369-139861127677616"
Jul 10 04:19:14.712436 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.712377    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b" containerName="test-curl-7369-139861127677616"
Jul 10 04:19:14.712476 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.712462    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b" containerName="test-curl-7369-139861127677616"
Jul 10 04:19:14.715489 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.715471    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:14.720964 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.720939    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7369-139861129640816"]
Jul 10 04:19:14.809087 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.809051    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m4rh\" (UniqueName: \"kubernetes.io/projected/552e0810-8ea8-4d45-af70-079117e418e9-kube-api-access-2m4rh\") pod \"test-curl-7369-139861129640816\" (UID: \"552e0810-8ea8-4d45-af70-079117e418e9\") " pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:14.910374 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.910278    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m4rh\" (UniqueName: \"kubernetes.io/projected/552e0810-8ea8-4d45-af70-079117e418e9-kube-api-access-2m4rh\") pod \"test-curl-7369-139861129640816\" (UID: \"552e0810-8ea8-4d45-af70-079117e418e9\") " pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:14.918727 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.918694    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m4rh\" (UniqueName: \"kubernetes.io/projected/552e0810-8ea8-4d45-af70-079117e418e9-kube-api-access-2m4rh\") pod \"test-curl-7369-139861129640816\" (UID: \"552e0810-8ea8-4d45-af70-079117e418e9\") " pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:14.994312 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:14.994279    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b" path="/var/lib/kubelet/pods/5ab33a3f-eeee-4ee1-b85b-5c3a17ec944b/volumes"
Jul 10 04:19:15.025922 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:15.025890    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:15.166383 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:15.166355    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7369-139861129640816"]
Jul 10 04:19:15.167787 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:19:15.167758    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod552e0810_8ea8_4d45_af70_079117e418e9.slice/crio-eb0e7fbb34e023f0cc21bee2b1482cff4cd1298d71cbc4a159073274da406530 WatchSource:0}: Error finding container eb0e7fbb34e023f0cc21bee2b1482cff4cd1298d71cbc4a159073274da406530: Status 404 returned error can't find the container with id eb0e7fbb34e023f0cc21bee2b1482cff4cd1298d71cbc4a159073274da406530
Jul 10 04:19:15.348033 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:15.347989    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7369-139861129640816" event={"ID":"552e0810-8ea8-4d45-af70-079117e418e9","Type":"ContainerStarted","Data":"eb0e7fbb34e023f0cc21bee2b1482cff4cd1298d71cbc4a159073274da406530"}
Jul 10 04:19:15.349297 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:15.349279    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861127677616"
Jul 10 04:19:15.349420 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:15.349332    2561 scope.go:117] "RemoveContainer" containerID="a4bc219d710045bd08095f4b71b6dd3d027c3b158de4f6b7a66f36be173b1939"
Jul 10 04:19:16.356947 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:16.356914    2561 generic.go:356] "Generic (PLEG): container finished" podID="552e0810-8ea8-4d45-af70-079117e418e9" containerID="c944304d72cff54bced9747e67ebaa55b6738dd9f5eb4fcb8067854fffc1b224" exitCode=0
Jul 10 04:19:16.357349 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:16.357001    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7369-139861129640816" event={"ID":"552e0810-8ea8-4d45-af70-079117e418e9","Type":"ContainerDied","Data":"c944304d72cff54bced9747e67ebaa55b6738dd9f5eb4fcb8067854fffc1b224"}
Jul 10 04:19:17.507152 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.507119    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:17.527343 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.527321    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7369-139861129640816_552e0810-8ea8-4d45-af70-079117e418e9/test-curl-7369-139861129640816/0.log"
Jul 10 04:19:17.529658 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.529621    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2m4rh\" (UniqueName: \"kubernetes.io/projected/552e0810-8ea8-4d45-af70-079117e418e9-kube-api-access-2m4rh\") pod \"552e0810-8ea8-4d45-af70-079117e418e9\" (UID: \"552e0810-8ea8-4d45-af70-079117e418e9\") "
Jul 10 04:19:17.531590 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.531563    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/552e0810-8ea8-4d45-af70-079117e418e9-kube-api-access-2m4rh" (OuterVolumeSpecName: "kube-api-access-2m4rh") pod "552e0810-8ea8-4d45-af70-079117e418e9" (UID: "552e0810-8ea8-4d45-af70-079117e418e9"). InnerVolumeSpecName "kube-api-access-2m4rh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:19:17.553436 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.553402    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7369-139861129640816"]
Jul 10 04:19:17.556951 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.556926    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7369-139861129640816"]
Jul 10 04:19:17.630389 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.630306    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2m4rh\" (UniqueName: \"kubernetes.io/projected/552e0810-8ea8-4d45-af70-079117e418e9-kube-api-access-2m4rh\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:17.687748 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.687708    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7369-139861129640816"]
Jul 10 04:19:17.688116 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.688104    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="552e0810-8ea8-4d45-af70-079117e418e9" containerName="test-curl-7369-139861129640816"
Jul 10 04:19:17.688175 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.688118    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="552e0810-8ea8-4d45-af70-079117e418e9" containerName="test-curl-7369-139861129640816"
Jul 10 04:19:17.688208 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.688184    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="552e0810-8ea8-4d45-af70-079117e418e9" containerName="test-curl-7369-139861129640816"
Jul 10 04:19:17.691294 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.691278    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:17.696403 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.696373    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7369-139861129640816"]
Jul 10 04:19:17.730697 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.730628    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7j44\" (UniqueName: \"kubernetes.io/projected/f3c506b1-dc60-4574-892b-324e7ef62dbc-kube-api-access-b7j44\") pod \"test-curl-7369-139861129640816\" (UID: \"f3c506b1-dc60-4574-892b-324e7ef62dbc\") " pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:17.831822 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.831789    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7j44\" (UniqueName: \"kubernetes.io/projected/f3c506b1-dc60-4574-892b-324e7ef62dbc-kube-api-access-b7j44\") pod \"test-curl-7369-139861129640816\" (UID: \"f3c506b1-dc60-4574-892b-324e7ef62dbc\") " pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:17.839676 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:17.839614    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7j44\" (UniqueName: \"kubernetes.io/projected/f3c506b1-dc60-4574-892b-324e7ef62dbc-kube-api-access-b7j44\") pod \"test-curl-7369-139861129640816\" (UID: \"f3c506b1-dc60-4574-892b-324e7ef62dbc\") " pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:18.001375 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:18.001276    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:18.139659 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:18.139621    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7369-139861129640816"]
Jul 10 04:19:18.140905 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:19:18.140879    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3c506b1_dc60_4574_892b_324e7ef62dbc.slice/crio-3b707ea30e1aba4ef0e1bb59f800a042697554a179f6bc2a1d85728067701998 WatchSource:0}: Error finding container 3b707ea30e1aba4ef0e1bb59f800a042697554a179f6bc2a1d85728067701998: Status 404 returned error can't find the container with id 3b707ea30e1aba4ef0e1bb59f800a042697554a179f6bc2a1d85728067701998
Jul 10 04:19:18.371442 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:18.371402    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7369-139861129640816" event={"ID":"f3c506b1-dc60-4574-892b-324e7ef62dbc","Type":"ContainerStarted","Data":"3b707ea30e1aba4ef0e1bb59f800a042697554a179f6bc2a1d85728067701998"}
Jul 10 04:19:18.376085 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:18.376059    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb0e7fbb34e023f0cc21bee2b1482cff4cd1298d71cbc4a159073274da406530"
Jul 10 04:19:18.376085 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:18.376071    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:18.388148 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:18.388116    2561 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7369-139861129640816" oldPodUID="552e0810-8ea8-4d45-af70-079117e418e9" podUID="f3c506b1-dc60-4574-892b-324e7ef62dbc"
Jul 10 04:19:18.994541 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:18.994512    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="552e0810-8ea8-4d45-af70-079117e418e9" path="/var/lib/kubelet/pods/552e0810-8ea8-4d45-af70-079117e418e9/volumes"
Jul 10 04:19:19.383669 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:19.383620    2561 generic.go:356] "Generic (PLEG): container finished" podID="f3c506b1-dc60-4574-892b-324e7ef62dbc" containerID="417aaccba9560f1b2314bf83e60577793b7a0624009f007cda095cc4662ff976" exitCode=0
Jul 10 04:19:19.383826 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:19.383682    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7369-139861129640816" event={"ID":"f3c506b1-dc60-4574-892b-324e7ef62dbc","Type":"ContainerDied","Data":"417aaccba9560f1b2314bf83e60577793b7a0624009f007cda095cc4662ff976"}
Jul 10 04:19:20.538421 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.538387    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:20.557008 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.556975    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7j44\" (UniqueName: \"kubernetes.io/projected/f3c506b1-dc60-4574-892b-324e7ef62dbc-kube-api-access-b7j44\") pod \"f3c506b1-dc60-4574-892b-324e7ef62dbc\" (UID: \"f3c506b1-dc60-4574-892b-324e7ef62dbc\") "
Jul 10 04:19:20.558347 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.558318    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7369-139861129640816_f3c506b1-dc60-4574-892b-324e7ef62dbc/test-curl-7369-139861129640816/0.log"
Jul 10 04:19:20.559231 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.559211    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3c506b1-dc60-4574-892b-324e7ef62dbc-kube-api-access-b7j44" (OuterVolumeSpecName: "kube-api-access-b7j44") pod "f3c506b1-dc60-4574-892b-324e7ef62dbc" (UID: "f3c506b1-dc60-4574-892b-324e7ef62dbc"). InnerVolumeSpecName "kube-api-access-b7j44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:19:20.582569 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.582543    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7369-139861129640816"]
Jul 10 04:19:20.585214 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.585193    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7369-139861129640816"]
Jul 10 04:19:20.657985 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.657904    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7j44\" (UniqueName: \"kubernetes.io/projected/f3c506b1-dc60-4574-892b-324e7ef62dbc-kube-api-access-b7j44\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:20.710078 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.710043    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7369-139861129641824"]
Jul 10 04:19:20.710468 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.710454    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3c506b1-dc60-4574-892b-324e7ef62dbc" containerName="test-curl-7369-139861129640816"
Jul 10 04:19:20.710527 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.710469    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3c506b1-dc60-4574-892b-324e7ef62dbc" containerName="test-curl-7369-139861129640816"
Jul 10 04:19:20.710566 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.710542    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3c506b1-dc60-4574-892b-324e7ef62dbc" containerName="test-curl-7369-139861129640816"
Jul 10 04:19:20.713672 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.713635    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:20.717371 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.717349    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7369-139861129641824"]
Jul 10 04:19:20.758750 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.758719    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdcvg\" (UniqueName: \"kubernetes.io/projected/f991e85c-6f9a-4612-8240-9fbcc024a578-kube-api-access-gdcvg\") pod \"test-curl-7369-139861129641824\" (UID: \"f991e85c-6f9a-4612-8240-9fbcc024a578\") " pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:20.859286 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.859241    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdcvg\" (UniqueName: \"kubernetes.io/projected/f991e85c-6f9a-4612-8240-9fbcc024a578-kube-api-access-gdcvg\") pod \"test-curl-7369-139861129641824\" (UID: \"f991e85c-6f9a-4612-8240-9fbcc024a578\") " pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:20.868933 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.868901    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdcvg\" (UniqueName: \"kubernetes.io/projected/f991e85c-6f9a-4612-8240-9fbcc024a578-kube-api-access-gdcvg\") pod \"test-curl-7369-139861129641824\" (UID: \"f991e85c-6f9a-4612-8240-9fbcc024a578\") " pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:20.994337 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:20.994259    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3c506b1-dc60-4574-892b-324e7ef62dbc" path="/var/lib/kubelet/pods/f3c506b1-dc60-4574-892b-324e7ef62dbc/volumes"
Jul 10 04:19:21.024043 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:21.024010    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:21.154611 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:21.154577    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7369-139861129641824"]
Jul 10 04:19:21.155600 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:19:21.155574    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf991e85c_6f9a_4612_8240_9fbcc024a578.slice/crio-511694bce8c7819b9c2c938323357b3478a0c3bec3f41337591854dd50e35af7 WatchSource:0}: Error finding container 511694bce8c7819b9c2c938323357b3478a0c3bec3f41337591854dd50e35af7: Status 404 returned error can't find the container with id 511694bce8c7819b9c2c938323357b3478a0c3bec3f41337591854dd50e35af7
Jul 10 04:19:21.396777 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:21.396737    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7369-139861129641824" event={"ID":"f991e85c-6f9a-4612-8240-9fbcc024a578","Type":"ContainerStarted","Data":"511694bce8c7819b9c2c938323357b3478a0c3bec3f41337591854dd50e35af7"}
Jul 10 04:19:21.398008 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:21.397979    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129640816"
Jul 10 04:19:21.398128 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:21.398038    2561 scope.go:117] "RemoveContainer" containerID="417aaccba9560f1b2314bf83e60577793b7a0624009f007cda095cc4662ff976"
Jul 10 04:19:22.405925 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:22.405892    2561 generic.go:356] "Generic (PLEG): container finished" podID="f991e85c-6f9a-4612-8240-9fbcc024a578" containerID="74c9ed3b4d4eb6ca095b2e83d14cfc99a1e2e739d641f73310cd6bc68b83786b" exitCode=0
Jul 10 04:19:22.406338 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:22.405975    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7369-139861129641824" event={"ID":"f991e85c-6f9a-4612-8240-9fbcc024a578","Type":"ContainerDied","Data":"74c9ed3b4d4eb6ca095b2e83d14cfc99a1e2e739d641f73310cd6bc68b83786b"}
Jul 10 04:19:23.547466 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.547439    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:23.571882 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.571854    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7369-139861129641824_f991e85c-6f9a-4612-8240-9fbcc024a578/test-curl-7369-139861129641824/0.log"
Jul 10 04:19:23.582864 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.582835    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdcvg\" (UniqueName: \"kubernetes.io/projected/f991e85c-6f9a-4612-8240-9fbcc024a578-kube-api-access-gdcvg\") pod \"f991e85c-6f9a-4612-8240-9fbcc024a578\" (UID: \"f991e85c-6f9a-4612-8240-9fbcc024a578\") "
Jul 10 04:19:23.584927 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.584900    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f991e85c-6f9a-4612-8240-9fbcc024a578-kube-api-access-gdcvg" (OuterVolumeSpecName: "kube-api-access-gdcvg") pod "f991e85c-6f9a-4612-8240-9fbcc024a578" (UID: "f991e85c-6f9a-4612-8240-9fbcc024a578"). InnerVolumeSpecName "kube-api-access-gdcvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:19:23.598612 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.598587    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7369-139861129641824"]
Jul 10 04:19:23.602135 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.602112    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7369-139861129641824"]
Jul 10 04:19:23.684457 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.684360    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdcvg\" (UniqueName: \"kubernetes.io/projected/f991e85c-6f9a-4612-8240-9fbcc024a578-kube-api-access-gdcvg\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:23.732723 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.732685    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7369-139861129641824"]
Jul 10 04:19:23.733097 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.733084    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f991e85c-6f9a-4612-8240-9fbcc024a578" containerName="test-curl-7369-139861129641824"
Jul 10 04:19:23.733145 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.733097    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f991e85c-6f9a-4612-8240-9fbcc024a578" containerName="test-curl-7369-139861129641824"
Jul 10 04:19:23.733184 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.733176    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f991e85c-6f9a-4612-8240-9fbcc024a578" containerName="test-curl-7369-139861129641824"
Jul 10 04:19:23.736270 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.736253    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:23.741349 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.741327    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7369-139861129641824"]
Jul 10 04:19:23.785565 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.785535    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mflg8\" (UniqueName: \"kubernetes.io/projected/038225ec-24b0-4843-962f-0a893b5417e1-kube-api-access-mflg8\") pod \"test-curl-7369-139861129641824\" (UID: \"038225ec-24b0-4843-962f-0a893b5417e1\") " pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:23.886873 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.886834    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mflg8\" (UniqueName: \"kubernetes.io/projected/038225ec-24b0-4843-962f-0a893b5417e1-kube-api-access-mflg8\") pod \"test-curl-7369-139861129641824\" (UID: \"038225ec-24b0-4843-962f-0a893b5417e1\") " pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:23.894305 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:23.894282    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mflg8\" (UniqueName: \"kubernetes.io/projected/038225ec-24b0-4843-962f-0a893b5417e1-kube-api-access-mflg8\") pod \"test-curl-7369-139861129641824\" (UID: \"038225ec-24b0-4843-962f-0a893b5417e1\") " pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:24.046741 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:24.046617    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:24.185803 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:24.185778    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7369-139861129641824"]
Jul 10 04:19:24.187272 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:19:24.187234    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod038225ec_24b0_4843_962f_0a893b5417e1.slice/crio-a82beaa2ed495773c98b7a7afbad6fe9fdae5d5bed9ec050fc7adad249da7ada WatchSource:0}: Error finding container a82beaa2ed495773c98b7a7afbad6fe9fdae5d5bed9ec050fc7adad249da7ada: Status 404 returned error can't find the container with id a82beaa2ed495773c98b7a7afbad6fe9fdae5d5bed9ec050fc7adad249da7ada
Jul 10 04:19:24.420341 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:24.420305    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7369-139861129641824" event={"ID":"038225ec-24b0-4843-962f-0a893b5417e1","Type":"ContainerStarted","Data":"a82beaa2ed495773c98b7a7afbad6fe9fdae5d5bed9ec050fc7adad249da7ada"}
Jul 10 04:19:24.421590 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:24.421568    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="511694bce8c7819b9c2c938323357b3478a0c3bec3f41337591854dd50e35af7"
Jul 10 04:19:24.421736 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:24.421597    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:24.432955 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:24.432929    2561 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7369-139861129641824" oldPodUID="f991e85c-6f9a-4612-8240-9fbcc024a578" podUID="038225ec-24b0-4843-962f-0a893b5417e1"
Jul 10 04:19:24.995182 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:24.995152    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f991e85c-6f9a-4612-8240-9fbcc024a578" path="/var/lib/kubelet/pods/f991e85c-6f9a-4612-8240-9fbcc024a578/volumes"
Jul 10 04:19:25.429625 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:25.429592    2561 generic.go:356] "Generic (PLEG): container finished" podID="038225ec-24b0-4843-962f-0a893b5417e1" containerID="866752f7b2573cae749222c934b82ff76ba1a4d40d6131c596b22633e730b86f" exitCode=0
Jul 10 04:19:25.429812 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:25.429675    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7369-139861129641824" event={"ID":"038225ec-24b0-4843-962f-0a893b5417e1","Type":"ContainerDied","Data":"866752f7b2573cae749222c934b82ff76ba1a4d40d6131c596b22633e730b86f"}
Jul 10 04:19:26.584765 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.584740    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:26.604257 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.604226    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7369-139861129641824_038225ec-24b0-4843-962f-0a893b5417e1/test-curl-7369-139861129641824/0.log"
Jul 10 04:19:26.611087 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.611062    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mflg8\" (UniqueName: \"kubernetes.io/projected/038225ec-24b0-4843-962f-0a893b5417e1-kube-api-access-mflg8\") pod \"038225ec-24b0-4843-962f-0a893b5417e1\" (UID: \"038225ec-24b0-4843-962f-0a893b5417e1\") "
Jul 10 04:19:26.613022 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.613003    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/038225ec-24b0-4843-962f-0a893b5417e1-kube-api-access-mflg8" (OuterVolumeSpecName: "kube-api-access-mflg8") pod "038225ec-24b0-4843-962f-0a893b5417e1" (UID: "038225ec-24b0-4843-962f-0a893b5417e1"). InnerVolumeSpecName "kube-api-access-mflg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:19:26.627393 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.627368    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7369-139861129641824"]
Jul 10 04:19:26.630870 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.630840    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7369-139861129641824"]
Jul 10 04:19:26.712494 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.712406    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mflg8\" (UniqueName: \"kubernetes.io/projected/038225ec-24b0-4843-962f-0a893b5417e1-kube-api-access-mflg8\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:26.825482 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.825440    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865"]
Jul 10 04:19:26.825950 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.825931    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038225ec-24b0-4843-962f-0a893b5417e1" containerName="test-curl-7369-139861129641824"
Jul 10 04:19:26.825950 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.825952    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="038225ec-24b0-4843-962f-0a893b5417e1" containerName="test-curl-7369-139861129641824"
Jul 10 04:19:26.826099 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.826049    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="038225ec-24b0-4843-962f-0a893b5417e1" containerName="test-curl-7369-139861129641824"
Jul 10 04:19:26.829152 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.829131    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865"
Jul 10 04:19:26.831252 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.831233    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:19:26.835684 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.835660    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865"]
Jul 10 04:19:26.914475 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.914444    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01eef285-4230-43fc-8d63-b254aa6bf647-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-937cca-th865\" (UID: \"01eef285-4230-43fc-8d63-b254aa6bf647\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865"
Jul 10 04:19:26.994379 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:26.994298    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="038225ec-24b0-4843-962f-0a893b5417e1" path="/var/lib/kubelet/pods/038225ec-24b0-4843-962f-0a893b5417e1/volumes"
Jul 10 04:19:27.015883 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:27.015847    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01eef285-4230-43fc-8d63-b254aa6bf647-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-937cca-th865\" (UID: \"01eef285-4230-43fc-8d63-b254aa6bf647\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865"
Jul 10 04:19:27.016436 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:27.016417    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01eef285-4230-43fc-8d63-b254aa6bf647-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-937cca-th865\" (UID: \"01eef285-4230-43fc-8d63-b254aa6bf647\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865"
Jul 10 04:19:27.142095 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:27.142061    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865"
Jul 10 04:19:27.285635 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:27.285605    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865"]
Jul 10 04:19:27.286669 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:19:27.286622    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01eef285_4230_43fc_8d63_b254aa6bf647.slice/crio-24292fbe98e2ad4744cccc4567dab65c37428d2c66bf1d1babe8e4f0ceb1e3e4 WatchSource:0}: Error finding container 24292fbe98e2ad4744cccc4567dab65c37428d2c66bf1d1babe8e4f0ceb1e3e4: Status 404 returned error can't find the container with id 24292fbe98e2ad4744cccc4567dab65c37428d2c66bf1d1babe8e4f0ceb1e3e4
Jul 10 04:19:27.303798 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:27.303747    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:27.404911 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:27.404880    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:27.445322 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:27.445286    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865" event={"ID":"01eef285-4230-43fc-8d63-b254aa6bf647","Type":"ContainerStarted","Data":"24292fbe98e2ad4744cccc4567dab65c37428d2c66bf1d1babe8e4f0ceb1e3e4"}
Jul 10 04:19:27.446632 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:27.446609    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7369-139861129641824"
Jul 10 04:19:27.446777 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:27.446677    2561 scope.go:117] "RemoveContainer" containerID="866752f7b2573cae749222c934b82ff76ba1a4d40d6131c596b22633e730b86f"
Jul 10 04:19:28.454875 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:28.454775    2561 generic.go:356] "Generic (PLEG): container finished" podID="01eef285-4230-43fc-8d63-b254aa6bf647" containerID="3a15dbbc04dfbad7cf422234dffd1574dc7f6b07eb77add55fe9001a22416bc3" exitCode=0
Jul 10 04:19:28.455270 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:28.454862    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865" event={"ID":"01eef285-4230-43fc-8d63-b254aa6bf647","Type":"ContainerDied","Data":"3a15dbbc04dfbad7cf422234dffd1574dc7f6b07eb77add55fe9001a22416bc3"}
Jul 10 04:19:29.603406 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:29.603384    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865"
Jul 10 04:19:29.637800 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:29.637766    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01eef285-4230-43fc-8d63-b254aa6bf647-maas-api-service-ca\") pod \"01eef285-4230-43fc-8d63-b254aa6bf647\" (UID: \"01eef285-4230-43fc-8d63-b254aa6bf647\") "
Jul 10 04:19:29.638134 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:29.638110    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01eef285-4230-43fc-8d63-b254aa6bf647-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01eef285-4230-43fc-8d63-b254aa6bf647" (UID: "01eef285-4230-43fc-8d63-b254aa6bf647"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:19:29.738862 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:29.738775    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01eef285-4230-43fc-8d63-b254aa6bf647-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:30.470999 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:30.470967    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865"
Jul 10 04:19:30.471165 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:30.470998    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-937cca-th865" event={"ID":"01eef285-4230-43fc-8d63-b254aa6bf647","Type":"ContainerDied","Data":"24292fbe98e2ad4744cccc4567dab65c37428d2c66bf1d1babe8e4f0ceb1e3e4"}
Jul 10 04:19:30.471165 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:30.471087    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24292fbe98e2ad4744cccc4567dab65c37428d2c66bf1d1babe8e4f0ceb1e3e4"
Jul 10 04:19:37.253217 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.253182    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"]
Jul 10 04:19:37.253679 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.253387    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh" podUID="6a55577d-8f8c-4ef7-bc32-e04840d08f09" containerName="maas-api" containerID="cri-o://52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661" gracePeriod=30
Jul 10 04:19:37.302238 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.302188    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:37.318868 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.318835    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh"]
Jul 10 04:19:37.329465 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.329439    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-937cca-29727615-r29qh"]
Jul 10 04:19:37.508941 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.508882    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:19:37.516245 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.516220    2561 generic.go:356] "Generic (PLEG): container finished" podID="6a55577d-8f8c-4ef7-bc32-e04840d08f09" containerID="52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661" exitCode=0
Jul 10 04:19:37.516353 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.516256    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"
Jul 10 04:19:37.516353 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.516298    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh" event={"ID":"6a55577d-8f8c-4ef7-bc32-e04840d08f09","Type":"ContainerDied","Data":"52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661"}
Jul 10 04:19:37.516428 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.516353    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh" event={"ID":"6a55577d-8f8c-4ef7-bc32-e04840d08f09","Type":"ContainerDied","Data":"52a09979b921d47a8a12f5a29fc42e5a9b3062cba48078ed0cff125f589c11aa"}
Jul 10 04:19:37.516428 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.516378    2561 scope.go:117] "RemoveContainer" containerID="52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661"
Jul 10 04:19:37.528380 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.528355    2561 scope.go:117] "RemoveContainer" containerID="52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661"
Jul 10 04:19:37.528691 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:19:37.528670    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661\": container with ID starting with 52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661 not found: ID does not exist" containerID="52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661"
Jul 10 04:19:37.528767 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.528701    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661"} err="failed to get container status \"52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661\": rpc error: code = NotFound desc = could not find container \"52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661\": container with ID starting with 52a9a82585ab9996adde450cfc001a8a693b4206e46f54ceaee5c44fc7dff661 not found: ID does not exist"
Jul 10 04:19:37.610307 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.610272    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vzs7\" (UniqueName: \"kubernetes.io/projected/6a55577d-8f8c-4ef7-bc32-e04840d08f09-kube-api-access-6vzs7\") pod \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") "
Jul 10 04:19:37.610307 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.610311    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a55577d-8f8c-4ef7-bc32-e04840d08f09-maas-api-tls\") pod \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") "
Jul 10 04:19:37.610555 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.610382    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a55577d-8f8c-4ef7-bc32-e04840d08f09-openshift-service-ca\") pod \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\" (UID: \"6a55577d-8f8c-4ef7-bc32-e04840d08f09\") "
Jul 10 04:19:37.610802 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.610778    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a55577d-8f8c-4ef7-bc32-e04840d08f09-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6a55577d-8f8c-4ef7-bc32-e04840d08f09" (UID: "6a55577d-8f8c-4ef7-bc32-e04840d08f09"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:19:37.612402 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.612379    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a55577d-8f8c-4ef7-bc32-e04840d08f09-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6a55577d-8f8c-4ef7-bc32-e04840d08f09" (UID: "6a55577d-8f8c-4ef7-bc32-e04840d08f09"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:19:37.612478 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.612463    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a55577d-8f8c-4ef7-bc32-e04840d08f09-kube-api-access-6vzs7" (OuterVolumeSpecName: "kube-api-access-6vzs7") pod "6a55577d-8f8c-4ef7-bc32-e04840d08f09" (UID: "6a55577d-8f8c-4ef7-bc32-e04840d08f09"). InnerVolumeSpecName "kube-api-access-6vzs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:19:37.711886 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.711853    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a55577d-8f8c-4ef7-bc32-e04840d08f09-openshift-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:37.711886 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.711880    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vzs7\" (UniqueName: \"kubernetes.io/projected/6a55577d-8f8c-4ef7-bc32-e04840d08f09-kube-api-access-6vzs7\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:37.711886 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.711890    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a55577d-8f8c-4ef7-bc32-e04840d08f09-maas-api-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:37.839097 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.839068    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"]
Jul 10 04:19:37.843070 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:37.843042    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-937cca-f9db68f9f-btzbh"]
Jul 10 04:19:38.995089 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:38.995057    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a55577d-8f8c-4ef7-bc32-e04840d08f09" path="/var/lib/kubelet/pods/6a55577d-8f8c-4ef7-bc32-e04840d08f09/volumes"
Jul 10 04:19:38.995538 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:38.995341    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeff4f42-fba8-4e06-b469-565eec2cdd1a" path="/var/lib/kubelet/pods/aeff4f42-fba8-4e06-b469-565eec2cdd1a/volumes"
Jul 10 04:19:41.912321 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:41.912286    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:47.881872 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:47.881828    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:47.935917 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:47.935885    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:52.587427 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.587391    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"]
Jul 10 04:19:52.587847 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.587774    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerName="istio-proxy" containerID="cri-o://43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7" gracePeriod=30
Jul 10 04:19:52.599697 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.599670    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:52.847842 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.847756    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:52.980128 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.980092    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p"]
Jul 10 04:19:52.980668 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.980632    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a55577d-8f8c-4ef7-bc32-e04840d08f09" containerName="maas-api"
Jul 10 04:19:52.980721 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.980667    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a55577d-8f8c-4ef7-bc32-e04840d08f09" containerName="maas-api"
Jul 10 04:19:52.980721 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.980693    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01eef285-4230-43fc-8d63-b254aa6bf647" containerName="revoke-keys"
Jul 10 04:19:52.980721 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.980703    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="01eef285-4230-43fc-8d63-b254aa6bf647" containerName="revoke-keys"
Jul 10 04:19:52.980831 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.980820    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a55577d-8f8c-4ef7-bc32-e04840d08f09" containerName="maas-api"
Jul 10 04:19:52.980867 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.980840    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="01eef285-4230-43fc-8d63-b254aa6bf647" containerName="revoke-keys"
Jul 10 04:19:52.986640 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.986611    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p"
Jul 10 04:19:52.988958 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.988932    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-spjdj\""
Jul 10 04:19:52.996441 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:52.996401    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p"]
Jul 10 04:19:53.154426 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:53.154335    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feefcfe3-8dc9-471e-acec-779e7c7f80f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p\" (UID: \"feefcfe3-8dc9-471e-acec-779e7c7f80f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p"
Jul 10 04:19:53.255338 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:53.255285    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feefcfe3-8dc9-471e-acec-779e7c7f80f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p\" (UID: \"feefcfe3-8dc9-471e-acec-779e7c7f80f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p"
Jul 10 04:19:53.256009 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:53.255979    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feefcfe3-8dc9-471e-acec-779e7c7f80f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p\" (UID: \"feefcfe3-8dc9-471e-acec-779e7c7f80f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p"
Jul 10 04:19:53.301269 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:53.301240    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p"
Jul 10 04:19:53.438869 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:53.438837    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:53.439629 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:53.439603    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p"]
Jul 10 04:19:53.441254 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:19:53.441229    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeefcfe3_8dc9_471e_acec_779e7c7f80f6.slice/crio-b7df7ee47aedea0acd3543c4c488b4fd4aeb95f59d5efba64be56173eaf946e4 WatchSource:0}: Error finding container b7df7ee47aedea0acd3543c4c488b4fd4aeb95f59d5efba64be56173eaf946e4: Status 404 returned error can't find the container with id b7df7ee47aedea0acd3543c4c488b4fd4aeb95f59d5efba64be56173eaf946e4
Jul 10 04:19:53.537830 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:53.537798    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:19:53.564425 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:53.564392    2561 patch_prober.go:28] interesting pod/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 04:19:53.564570 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:53.564477    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 04:19:53.618436 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:53.618396    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p" event={"ID":"feefcfe3-8dc9-471e-acec-779e7c7f80f6","Type":"ContainerStarted","Data":"b7df7ee47aedea0acd3543c4c488b4fd4aeb95f59d5efba64be56173eaf946e4"}
Jul 10 04:19:54.625735 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:54.625698    2561 generic.go:356] "Generic (PLEG): container finished" podID="feefcfe3-8dc9-471e-acec-779e7c7f80f6" containerID="b65afde191cdb297e34dc032beaa6f270488b86408d3fd0f8593c460753a06c8" exitCode=0
Jul 10 04:19:54.626121 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:54.625781    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p" event={"ID":"feefcfe3-8dc9-471e-acec-779e7c7f80f6","Type":"ContainerDied","Data":"b65afde191cdb297e34dc032beaa6f270488b86408d3fd0f8593c460753a06c8"}
Jul 10 04:19:55.776810 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:55.776783    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p"
Jul 10 04:19:55.881813 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:55.881779    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feefcfe3-8dc9-471e-acec-779e7c7f80f6-maas-api-service-ca\") pod \"feefcfe3-8dc9-471e-acec-779e7c7f80f6\" (UID: \"feefcfe3-8dc9-471e-acec-779e7c7f80f6\") "
Jul 10 04:19:55.882162 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:55.882138    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feefcfe3-8dc9-471e-acec-779e7c7f80f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "feefcfe3-8dc9-471e-acec-779e7c7f80f6" (UID: "feefcfe3-8dc9-471e-acec-779e7c7f80f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:19:55.983007 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:55.982922    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feefcfe3-8dc9-471e-acec-779e7c7f80f6-maas-api-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:56.639280 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:56.639248    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p"
Jul 10 04:19:56.639440 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:56.639253    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p" event={"ID":"feefcfe3-8dc9-471e-acec-779e7c7f80f6","Type":"ContainerDied","Data":"b7df7ee47aedea0acd3543c4c488b4fd4aeb95f59d5efba64be56173eaf946e4"}
Jul 10 04:19:56.639440 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:56.639364    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7df7ee47aedea0acd3543c4c488b4fd4aeb95f59d5efba64be56173eaf946e4"
Jul 10 04:19:57.857048 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:57.856988    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:19:58.001854 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.001819    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-data\") pod \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") "
Jul 10 04:19:58.002032 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.001868    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-workload-certs\") pod \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") "
Jul 10 04:19:58.002032 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.001894    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-workload-socket\") pod \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") "
Jul 10 04:19:58.002032 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.001936    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-envoy\") pod \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") "
Jul 10 04:19:58.002032 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.001976    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-credential-socket\") pod \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") "
Jul 10 04:19:58.002032 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.002024    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istiod-ca-cert\") pod \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") "
Jul 10 04:19:58.002293 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.002068    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdd4x\" (UniqueName: \"kubernetes.io/projected/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-kube-api-access-pdd4x\") pod \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") "
Jul 10 04:19:58.002293 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.002111    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-podinfo\") pod \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") "
Jul 10 04:19:58.002293 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.002140    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-token\") pod \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\" (UID: \"7d513a68-dbe4-4c28-b515-8dd45f8b26d4\") "
Jul 10 04:19:58.002293 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.002178    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7d513a68-dbe4-4c28-b515-8dd45f8b26d4" (UID: "7d513a68-dbe4-4c28-b515-8dd45f8b26d4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:58.002293 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.002249    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7d513a68-dbe4-4c28-b515-8dd45f8b26d4" (UID: "7d513a68-dbe4-4c28-b515-8dd45f8b26d4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:58.002569 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.002473    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7d513a68-dbe4-4c28-b515-8dd45f8b26d4" (UID: "7d513a68-dbe4-4c28-b515-8dd45f8b26d4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:58.002569 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.002495    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-workload-certs\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:58.002569 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.002513    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-workload-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:58.002769 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.002636    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7d513a68-dbe4-4c28-b515-8dd45f8b26d4" (UID: "7d513a68-dbe4-4c28-b515-8dd45f8b26d4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:19:58.002830 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.002769    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-data" (OuterVolumeSpecName: "istio-data") pod "7d513a68-dbe4-4c28-b515-8dd45f8b26d4" (UID: "7d513a68-dbe4-4c28-b515-8dd45f8b26d4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:58.004376 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.004347    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7d513a68-dbe4-4c28-b515-8dd45f8b26d4" (UID: "7d513a68-dbe4-4c28-b515-8dd45f8b26d4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 04:19:58.004502 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.004403    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-kube-api-access-pdd4x" (OuterVolumeSpecName: "kube-api-access-pdd4x") pod "7d513a68-dbe4-4c28-b515-8dd45f8b26d4" (UID: "7d513a68-dbe4-4c28-b515-8dd45f8b26d4"). InnerVolumeSpecName "kube-api-access-pdd4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:19:58.004564 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.004550    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-token" (OuterVolumeSpecName: "istio-token") pod "7d513a68-dbe4-4c28-b515-8dd45f8b26d4" (UID: "7d513a68-dbe4-4c28-b515-8dd45f8b26d4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:19:58.004564 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.004553    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7d513a68-dbe4-4c28-b515-8dd45f8b26d4" (UID: "7d513a68-dbe4-4c28-b515-8dd45f8b26d4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:58.103789 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.103763    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-podinfo\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:58.103789 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.103789    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-token\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:58.103954 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.103798    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-data\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:58.103954 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.103808    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istio-envoy\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:58.103954 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.103816    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-credential-socket\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:58.103954 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.103824    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-istiod-ca-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:58.103954 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.103832    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdd4x\" (UniqueName: \"kubernetes.io/projected/7d513a68-dbe4-4c28-b515-8dd45f8b26d4-kube-api-access-pdd4x\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:19:58.652534 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.652502    2561 generic.go:356] "Generic (PLEG): container finished" podID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerID="43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7" exitCode=0
Jul 10 04:19:58.652741 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.652550    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"
Jul 10 04:19:58.652741 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.652595    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" event={"ID":"7d513a68-dbe4-4c28-b515-8dd45f8b26d4","Type":"ContainerDied","Data":"43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7"}
Jul 10 04:19:58.652741 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.652674    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4" event={"ID":"7d513a68-dbe4-4c28-b515-8dd45f8b26d4","Type":"ContainerDied","Data":"6b7099e9eed9919e657b38b9a8fa435a54518ee6004ba0ef412c2f077bd79698"}
Jul 10 04:19:58.652741 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.652700    2561 scope.go:117] "RemoveContainer" containerID="43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7"
Jul 10 04:19:58.662629 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.662613    2561 scope.go:117] "RemoveContainer" containerID="96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a"
Jul 10 04:19:58.671595 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.671573    2561 scope.go:117] "RemoveContainer" containerID="43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7"
Jul 10 04:19:58.671942 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:19:58.671912    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7\": container with ID starting with 43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7 not found: ID does not exist" containerID="43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7"
Jul 10 04:19:58.672051 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.671952    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7"} err="failed to get container status \"43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7\": rpc error: code = NotFound desc = could not find container \"43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7\": container with ID starting with 43e0ff610881ca4fd403d4624efdd689d75b31786c45edb27fba2f31038a92b7 not found: ID does not exist"
Jul 10 04:19:58.672051 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.671976    2561 scope.go:117] "RemoveContainer" containerID="96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a"
Jul 10 04:19:58.672224 ip-10-0-140-167 kubenswrapper[2561]: E0710 04:19:58.672196    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a\": container with ID starting with 96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a not found: ID does not exist" containerID="96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a"
Jul 10 04:19:58.672318 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.672223    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a"} err="failed to get container status \"96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a\": rpc error: code = NotFound desc = could not find container \"96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a\": container with ID starting with 96c1358108dd88dd2471ae91668aedd19fd6291a6a9f679959025ad1cdc2665a not found: ID does not exist"
Jul 10 04:19:58.674537 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.674516    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"]
Jul 10 04:19:58.678638 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.678618    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-937cca-openshift-default-fb8d5bbf5-62cm4"]
Jul 10 04:19:58.994016 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:19:58.993937    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" path="/var/lib/kubelet/pods/7d513a68-dbe4-4c28-b515-8dd45f8b26d4/volumes"
Jul 10 04:20:03.329129 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.329095    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"]
Jul 10 04:20:03.329524 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.329335    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d" podUID="98fec6aa-0cf6-4202-a074-417e0b6f8665" containerName="maas-api" containerID="cri-o://c51d40940a849ee4f06a2ba342a505a86db611da2c2734ebc1c552441e483183" gracePeriod=30
Jul 10 04:20:03.352320 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.352286    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3505cb-29727615-jp4pt"]
Jul 10 04:20:03.354819 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.354795    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3505cb-29727615-jp4pt"]
Jul 10 04:20:03.361759 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.361735    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:20:03.685946 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.685915    2561 generic.go:356] "Generic (PLEG): container finished" podID="98fec6aa-0cf6-4202-a074-417e0b6f8665" containerID="c51d40940a849ee4f06a2ba342a505a86db611da2c2734ebc1c552441e483183" exitCode=0
Jul 10 04:20:03.686109 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.685995    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d" event={"ID":"98fec6aa-0cf6-4202-a074-417e0b6f8665","Type":"ContainerDied","Data":"c51d40940a849ee4f06a2ba342a505a86db611da2c2734ebc1c552441e483183"}
Jul 10 04:20:03.785824 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.785799    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:20:03.852750 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.852668    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98fec6aa-0cf6-4202-a074-417e0b6f8665-openshift-service-ca\") pod \"98fec6aa-0cf6-4202-a074-417e0b6f8665\" (UID: \"98fec6aa-0cf6-4202-a074-417e0b6f8665\") "
Jul 10 04:20:03.852750 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.852727    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdj2c\" (UniqueName: \"kubernetes.io/projected/98fec6aa-0cf6-4202-a074-417e0b6f8665-kube-api-access-rdj2c\") pod \"98fec6aa-0cf6-4202-a074-417e0b6f8665\" (UID: \"98fec6aa-0cf6-4202-a074-417e0b6f8665\") "
Jul 10 04:20:03.852933 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.852769    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98fec6aa-0cf6-4202-a074-417e0b6f8665-maas-api-tls\") pod \"98fec6aa-0cf6-4202-a074-417e0b6f8665\" (UID: \"98fec6aa-0cf6-4202-a074-417e0b6f8665\") "
Jul 10 04:20:03.853030 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.853007    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98fec6aa-0cf6-4202-a074-417e0b6f8665-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98fec6aa-0cf6-4202-a074-417e0b6f8665" (UID: "98fec6aa-0cf6-4202-a074-417e0b6f8665"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:20:03.854925 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.854894    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98fec6aa-0cf6-4202-a074-417e0b6f8665-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98fec6aa-0cf6-4202-a074-417e0b6f8665" (UID: "98fec6aa-0cf6-4202-a074-417e0b6f8665"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:20:03.855040 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.855009    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98fec6aa-0cf6-4202-a074-417e0b6f8665-kube-api-access-rdj2c" (OuterVolumeSpecName: "kube-api-access-rdj2c") pod "98fec6aa-0cf6-4202-a074-417e0b6f8665" (UID: "98fec6aa-0cf6-4202-a074-417e0b6f8665"). InnerVolumeSpecName "kube-api-access-rdj2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:20:03.953567 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.953532    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98fec6aa-0cf6-4202-a074-417e0b6f8665-maas-api-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:20:03.953567 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.953561    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98fec6aa-0cf6-4202-a074-417e0b6f8665-openshift-service-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:20:03.953567 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:03.953574    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdj2c\" (UniqueName: \"kubernetes.io/projected/98fec6aa-0cf6-4202-a074-417e0b6f8665-kube-api-access-rdj2c\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\""
Jul 10 04:20:04.693970 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:04.693940    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"
Jul 10 04:20:04.693970 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:04.693958    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d" event={"ID":"98fec6aa-0cf6-4202-a074-417e0b6f8665","Type":"ContainerDied","Data":"f4d0107229d1ad925720988ac1e2d07d9e99734f972a9d756bd94450ebd726a7"}
Jul 10 04:20:04.694458 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:04.694021    2561 scope.go:117] "RemoveContainer" containerID="c51d40940a849ee4f06a2ba342a505a86db611da2c2734ebc1c552441e483183"
Jul 10 04:20:04.715209 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:04.715184    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"]
Jul 10 04:20:04.720386 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:04.720363    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3505cb-7cc5f4f486-m682d"]
Jul 10 04:20:04.994139 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:04.994056    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="308028de-939b-4e66-ab72-fb9261cce6b0" path="/var/lib/kubelet/pods/308028de-939b-4e66-ab72-fb9261cce6b0/volumes"
Jul 10 04:20:04.994445 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:04.994431    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98fec6aa-0cf6-4202-a074-417e0b6f8665" path="/var/lib/kubelet/pods/98fec6aa-0cf6-4202-a074-417e0b6f8665/volumes"
Jul 10 04:20:08.052002 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:08.051965    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:20:13.727747 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:13.727666    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:20:13.781910 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:13.781877    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:20:18.788672 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:18.788624    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:20:19.042378 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:19.042291    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:20:19.489573 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:19.489550    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6999f8c9db-jqmxt_610204f2-9096-426d-bff7-3be22c784ec2/authorino/0.log"
Jul 10 04:20:24.756440 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:24.756405    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f864f4b59-9h9qh_dc45a6d4-dac9-497f-91b9-873e433b0dde/manager/0.log"
Jul 10 04:20:25.268616 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:25.268588    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d676bbf4b-grr6m_822c49c8-42d2-42f5-9684-b21afcdc591f/copy-manifests/0.log"
Jul 10 04:20:25.275615 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:25.275590    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d676bbf4b-grr6m_822c49c8-42d2-42f5-9684-b21afcdc591f/manager/0.log"
Jul 10 04:20:25.922942 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:25.922913    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29727585-7bh42_fe7272ca-08d5-4953-96bd-f9215d37d411/cleanup/0.log"
Jul 10 04:20:26.043851 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:26.043824    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29727600-qjvxv_d1c68fd1-41e4-460b-9b40-1c1d66d08bce/cleanup/0.log"
Jul 10 04:20:26.159071 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:26.159038    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29727615-t4jpr_ed760aa2-906e-4fa6-87f8-11b09ef5e807/cleanup/0.log"
Jul 10 04:20:26.274644 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:26.274569    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-11393920-g45c6_bb5b1e0e-5e7b-4ac6-bd0b-20e18f2e0d9a/revoke-keys/0.log"
Jul 10 04:20:26.393548 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:26.393520    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-8d8a2105-dh6xs_89d9a6a4-c002-465d-a30e-e6909f228104/revoke-keys/0.log"
Jul 10 04:20:26.511493 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:26.511463    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-f48e73-lnvf7_418240d0-3383-4ffe-a8b7-6ccce10de838/revoke-keys/0.log"
Jul 10 04:20:26.628520 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:26.628489    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-a335e2-bzw5z_29978b1d-4bb8-48dc-8056-1ada76651e6e/revoke-keys/0.log"
Jul 10 04:20:26.749669 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:26.749614    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-6b62d0e3-bfqdt_50116307-d0ea-414c-80f0-dca5d287a0ff/revoke-keys/0.log"
Jul 10 04:20:26.869348 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:26.869321    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-72703e-nzb9f_4c95d1a6-ba37-45e5-9e4a-26e1d8a09ad6/revoke-keys/0.log"
Jul 10 04:20:26.993146 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:26.993067    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-7fe142-wgxtn_8818641a-d1eb-4bd2-87c0-66105ff629cc/revoke-keys/0.log"
Jul 10 04:20:27.116442 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:27.116406    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-010d76e9-n74qt_e7064bb2-f33e-40d1-9590-96c63a285dc6/revoke-keys/0.log"
Jul 10 04:20:27.236873 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:27.236838    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-aa8dc0f1-hpzhn_a8f57935-fd31-47bd-80a4-f2c0a21f98d9/revoke-keys/0.log"
Jul 10 04:20:27.365488 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:27.365459    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-937cca-th865_01eef285-4230-43fc-8d63-b254aa6bf647/revoke-keys/0.log"
Jul 10 04:20:27.491317 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:27.491292    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-3505cb-bmq2p_feefcfe3-8dc9-471e-acec-779e7c7f80f6/revoke-keys/0.log"
Jul 10 04:20:28.443325 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.443295    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq_ed03f018-31e5-42e3-9ed0-f676d890a384/extract/0.log"
Jul 10 04:20:28.449802 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.449777    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq_ed03f018-31e5-42e3-9ed0-f676d890a384/util/0.log"
Jul 10 04:20:28.456721 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.456700    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq_ed03f018-31e5-42e3-9ed0-f676d890a384/pull/0.log"
Jul 10 04:20:28.571774 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.571744    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq_d5fedb8f-fbc2-4244-921c-5b12e9790b0c/util/0.log"
Jul 10 04:20:28.578289 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.578268    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq_d5fedb8f-fbc2-4244-921c-5b12e9790b0c/pull/0.log"
Jul 10 04:20:28.584496 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.584478    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq_d5fedb8f-fbc2-4244-921c-5b12e9790b0c/extract/0.log"
Jul 10 04:20:28.707676 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.707566    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r_bb5e9022-8d73-46cc-9a8a-1c026c307c69/util/0.log"
Jul 10 04:20:28.714484 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.714454    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r_bb5e9022-8d73-46cc-9a8a-1c026c307c69/pull/0.log"
Jul 10 04:20:28.720956 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.720933    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r_bb5e9022-8d73-46cc-9a8a-1c026c307c69/extract/0.log"
Jul 10 04:20:28.852541 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.852515    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4_87c78c13-e85e-4a75-a4f8-d351da005359/util/0.log"
Jul 10 04:20:28.859158 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.859136    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4_87c78c13-e85e-4a75-a4f8-d351da005359/pull/0.log"
Jul 10 04:20:28.866097 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.866076    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4_87c78c13-e85e-4a75-a4f8-d351da005359/extract/0.log"
Jul 10 04:20:28.995620 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:28.995539    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6999f8c9db-jqmxt_610204f2-9096-426d-bff7-3be22c784ec2/authorino/0.log"
Jul 10 04:20:29.346765 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:29.346731    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-wbcrq_197c9c66-7a0e-4317-a816-b6db7422d5d6/kuadrant-console-plugin/0.log"
Jul 10 04:20:30.562333 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:30.562302    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d9645bfdd-gmc4w_dfca196d-b171-4d6c-8989-91ff58a6ffd5/kube-auth-proxy/0.log"
Jul 10 04:20:30.806172 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:30.806144    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-rnqxm_11c1301b-ab4e-490b-8012-e973bd2b00be/payload-pre-processing/0.log"
Jul 10 04:20:30.923049 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:30.923019    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ldtg2_fb1e2f13-1834-43a2-8f83-786caa86eaa2/payload-processing/0.log"
Jul 10 04:20:35.823489 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823454    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pg24g/must-gather-z7l76"]
Jul 10 04:20:35.823881 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823840    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerName="istio-proxy"
Jul 10 04:20:35.823881 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823852    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerName="istio-proxy"
Jul 10 04:20:35.823881 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823864    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerName="istio-proxy"
Jul 10 04:20:35.823881 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823870    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerName="istio-proxy"
Jul 10 04:20:35.823881 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823883    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98fec6aa-0cf6-4202-a074-417e0b6f8665" containerName="maas-api"
Jul 10 04:20:35.824044 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823889    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="98fec6aa-0cf6-4202-a074-417e0b6f8665" containerName="maas-api"
Jul 10 04:20:35.824044 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823901    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feefcfe3-8dc9-471e-acec-779e7c7f80f6" containerName="revoke-keys"
Jul 10 04:20:35.824044 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823907    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="feefcfe3-8dc9-471e-acec-779e7c7f80f6" containerName="revoke-keys"
Jul 10 04:20:35.824044 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823968    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerName="istio-proxy"
Jul 10 04:20:35.824044 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823975    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d513a68-dbe4-4c28-b515-8dd45f8b26d4" containerName="istio-proxy"
Jul 10 04:20:35.824044 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823982    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="feefcfe3-8dc9-471e-acec-779e7c7f80f6" containerName="revoke-keys"
Jul 10 04:20:35.824044 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.823988    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="98fec6aa-0cf6-4202-a074-417e0b6f8665" containerName="maas-api"
Jul 10 04:20:35.827283 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.827267    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pg24g/must-gather-z7l76"
Jul 10 04:20:35.829848 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.829826    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pg24g\"/\"default-dockercfg-96jld\""
Jul 10 04:20:35.829987 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.829828    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pg24g\"/\"openshift-service-ca.crt\""
Jul 10 04:20:35.830565 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.830551    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pg24g\"/\"kube-root-ca.crt\""
Jul 10 04:20:35.833570 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.833538    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:20:35.836272 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.836250    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pg24g/must-gather-z7l76"]
Jul 10 04:20:35.914313 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.914276    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2b48d32e-4aca-4086-9192-bc200136f589-must-gather-output\") pod \"must-gather-z7l76\" (UID: \"2b48d32e-4aca-4086-9192-bc200136f589\") " pod="openshift-must-gather-pg24g/must-gather-z7l76"
Jul 10 04:20:35.914491 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:35.914318    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9jgq\" (UniqueName: \"kubernetes.io/projected/2b48d32e-4aca-4086-9192-bc200136f589-kube-api-access-t9jgq\") pod \"must-gather-z7l76\" (UID: \"2b48d32e-4aca-4086-9192-bc200136f589\") " pod="openshift-must-gather-pg24g/must-gather-z7l76"
Jul 10 04:20:36.015676 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:36.015621    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2b48d32e-4aca-4086-9192-bc200136f589-must-gather-output\") pod \"must-gather-z7l76\" (UID: \"2b48d32e-4aca-4086-9192-bc200136f589\") " pod="openshift-must-gather-pg24g/must-gather-z7l76"
Jul 10 04:20:36.015849 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:36.015691    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9jgq\" (UniqueName: \"kubernetes.io/projected/2b48d32e-4aca-4086-9192-bc200136f589-kube-api-access-t9jgq\") pod \"must-gather-z7l76\" (UID: \"2b48d32e-4aca-4086-9192-bc200136f589\") " pod="openshift-must-gather-pg24g/must-gather-z7l76"
Jul 10 04:20:36.015978 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:36.015957    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2b48d32e-4aca-4086-9192-bc200136f589-must-gather-output\") pod \"must-gather-z7l76\" (UID: \"2b48d32e-4aca-4086-9192-bc200136f589\") " pod="openshift-must-gather-pg24g/must-gather-z7l76"
Jul 10 04:20:36.025078 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:36.025052    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9jgq\" (UniqueName: \"kubernetes.io/projected/2b48d32e-4aca-4086-9192-bc200136f589-kube-api-access-t9jgq\") pod \"must-gather-z7l76\" (UID: \"2b48d32e-4aca-4086-9192-bc200136f589\") " pod="openshift-must-gather-pg24g/must-gather-z7l76"
Jul 10 04:20:36.138903 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:36.138804    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pg24g/must-gather-z7l76"
Jul 10 04:20:36.275696 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:36.275665    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pg24g/must-gather-z7l76"]
Jul 10 04:20:36.276406 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:20:36.276380    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b48d32e_4aca_4086_9192_bc200136f589.slice/crio-867b4a1207f216a4dbc70fdf1b750365fb0483d99c537b12b6663fe74ad752b4 WatchSource:0}: Error finding container 867b4a1207f216a4dbc70fdf1b750365fb0483d99c537b12b6663fe74ad752b4: Status 404 returned error can't find the container with id 867b4a1207f216a4dbc70fdf1b750365fb0483d99c537b12b6663fe74ad752b4
Jul 10 04:20:36.889914 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:36.889876    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pg24g/must-gather-z7l76" event={"ID":"2b48d32e-4aca-4086-9192-bc200136f589","Type":"ContainerStarted","Data":"867b4a1207f216a4dbc70fdf1b750365fb0483d99c537b12b6663fe74ad752b4"}
Jul 10 04:20:36.969623 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:36.969585    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log" limit=1024
Jul 10 04:20:37.899956 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:37.899921    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pg24g/must-gather-z7l76" event={"ID":"2b48d32e-4aca-4086-9192-bc200136f589","Type":"ContainerStarted","Data":"dd3d972d63b6e1f0325cdfc9b719c985b3c94606cc3ae4d07174db510a58fda3"}
Jul 10 04:20:37.899956 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:37.899968    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pg24g/must-gather-z7l76" event={"ID":"2b48d32e-4aca-4086-9192-bc200136f589","Type":"ContainerStarted","Data":"05fc75284638ae2f2bc53dc080975a5c472a7bf0aee7364028b0cff0255c812f"}
Jul 10 04:20:37.914878 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:37.914821    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pg24g/must-gather-z7l76" podStartSLOduration=2.083444479 podStartE2EDuration="2.914805903s" totalImagesPullingTime="831.361424ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:20:37.914012793 +0000 UTC m=+2899.405008668" watchObservedRunningTime="2026-07-10 04:20:37.914805903 +0000 UTC m=+2899.405801818"
Jul 10 04:20:38.656990 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:38.656961    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lzrdd_ed5febd9-f003-483e-bd13-02cf0eeb73b6/global-pull-secret-syncer/0.log"
Jul 10 04:20:38.751404 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:38.751369    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-47lw8_1e21a35d-82a3-4576-8a55-207ebdc1595e/konnectivity-agent/0.log"
Jul 10 04:20:38.859863 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:38.859834    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-167.ec2.internal_d6cf1273741a1c23d1af7ea6ad5bb863/haproxy/0.log"
Jul 10 04:20:43.109549 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.109519    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq_ed03f018-31e5-42e3-9ed0-f676d890a384/extract/0.log"
Jul 10 04:20:43.137819 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.137791    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq_ed03f018-31e5-42e3-9ed0-f676d890a384/util/0.log"
Jul 10 04:20:43.172875 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.172846    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tb7nq_ed03f018-31e5-42e3-9ed0-f676d890a384/pull/0.log"
Jul 10 04:20:43.204233 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.204205    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq_d5fedb8f-fbc2-4244-921c-5b12e9790b0c/extract/0.log"
Jul 10 04:20:43.234055 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.234029    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq_d5fedb8f-fbc2-4244-921c-5b12e9790b0c/util/0.log"
Jul 10 04:20:43.264625 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.264598    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fwpjq_d5fedb8f-fbc2-4244-921c-5b12e9790b0c/pull/0.log"
Jul 10 04:20:43.298271 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.298227    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r_bb5e9022-8d73-46cc-9a8a-1c026c307c69/extract/0.log"
Jul 10 04:20:43.321531 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.321503    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r_bb5e9022-8d73-46cc-9a8a-1c026c307c69/util/0.log"
Jul 10 04:20:43.346287 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.346245    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hz7r_bb5e9022-8d73-46cc-9a8a-1c026c307c69/pull/0.log"
Jul 10 04:20:43.383618 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.383524    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4_87c78c13-e85e-4a75-a4f8-d351da005359/extract/0.log"
Jul 10 04:20:43.404973 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.404885    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4_87c78c13-e85e-4a75-a4f8-d351da005359/util/0.log"
Jul 10 04:20:43.426264 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.426200    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c4zp4_87c78c13-e85e-4a75-a4f8-d351da005359/pull/0.log"
Jul 10 04:20:43.801445 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.801070    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6999f8c9db-jqmxt_610204f2-9096-426d-bff7-3be22c784ec2/authorino/0.log"
Jul 10 04:20:43.887525 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:43.887491    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-wbcrq_197c9c66-7a0e-4317-a816-b6db7422d5d6/kuadrant-console-plugin/0.log"
Jul 10 04:20:46.079815 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.079787    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7b9d8fb85f-mpsmw_018616ab-fbf6-4171-bf25-c39c8b8446d4/metrics-server/0.log"
Jul 10 04:20:46.114531 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.114499    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-7gmsg_a3cdab6f-eed4-4108-93fd-0231e7c3b4ff/monitoring-plugin/0.log"
Jul 10 04:20:46.319666 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.319562    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v49v2_81c9f818-0f55-425e-a28e-a921feecbdf5/node-exporter/0.log"
Jul 10 04:20:46.336059 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.336024    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v49v2_81c9f818-0f55-425e-a28e-a921feecbdf5/kube-rbac-proxy/0.log"
Jul 10 04:20:46.358788 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.358756    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v49v2_81c9f818-0f55-425e-a28e-a921feecbdf5/init-textfile/0.log"
Jul 10 04:20:46.645971 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.645934    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-bmjvn_ee46ee9f-caae-4525-a907-ccc8065de209/prometheus-operator/0.log"
Jul 10 04:20:46.669145 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.669111    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-bmjvn_ee46ee9f-caae-4525-a907-ccc8065de209/kube-rbac-proxy/0.log"
Jul 10 04:20:46.826081 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.826054    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7cb5995774-8lqs7_293c39d2-6ccb-49b0-adee-2c3952399b07/thanos-query/0.log"
Jul 10 04:20:46.846425 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.846361    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7cb5995774-8lqs7_293c39d2-6ccb-49b0-adee-2c3952399b07/kube-rbac-proxy-web/0.log"
Jul 10 04:20:46.868432 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.868404    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7cb5995774-8lqs7_293c39d2-6ccb-49b0-adee-2c3952399b07/kube-rbac-proxy/0.log"
Jul 10 04:20:46.888830 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.888804    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7cb5995774-8lqs7_293c39d2-6ccb-49b0-adee-2c3952399b07/prom-label-proxy/0.log"
Jul 10 04:20:46.913421 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.913318    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7cb5995774-8lqs7_293c39d2-6ccb-49b0-adee-2c3952399b07/kube-rbac-proxy-rules/0.log"
Jul 10 04:20:46.933622 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:46.933592    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7cb5995774-8lqs7_293c39d2-6ccb-49b0-adee-2c3952399b07/kube-rbac-proxy-metrics/0.log"
Jul 10 04:20:48.109830 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:48.109782    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-9gjc5_ed31fb70-7e2b-4d84-946b-b00431c7e731/networking-console-plugin/0.log"
Jul 10 04:20:48.734734 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:48.734697    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/1.log"
Jul 10 04:20:48.741973 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:48.741945    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dcjt9_b41fb4a3-4b8d-4c2d-b751-2283a10b4234/console-operator/2.log"
Jul 10 04:20:49.274571 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.274535    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7fd8999687-gn9xg_d825549b-7e39-46ee-a09f-d9323079ee6b/console/0.log"
Jul 10 04:20:49.311976 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.311944    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-l8l4q_9900998f-2e50-4777-867d-137431c31b88/download-server/0.log"
Jul 10 04:20:49.376401 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.376359    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"]
Jul 10 04:20:49.382891 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.382867    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.390778 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.390751    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"]
Jul 10 04:20:49.459704 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.459339    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-podres\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.459704 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.459410    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-sys\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.459704 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.459465    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-proc\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.459704 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.459495    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-lib-modules\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.459704 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.459529    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghmjp\" (UniqueName: \"kubernetes.io/projected/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-kube-api-access-ghmjp\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.560547 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.560440    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-podres\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.561304 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.560640    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-podres\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.561304 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.560825    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-sys\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.561304 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.560884    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-sys\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.561304 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.560914    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-proc\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.561304 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.560947    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-lib-modules\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.561304 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.560983    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghmjp\" (UniqueName: \"kubernetes.io/projected/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-kube-api-access-ghmjp\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.561304 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.561035    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-proc\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.561304 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.561164    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-lib-modules\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.570716 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.570689    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghmjp\" (UniqueName: \"kubernetes.io/projected/4cbf92da-02e7-4c75-9e9c-715d0d05f0ff-kube-api-access-ghmjp\") pod \"perf-node-gather-daemonset-zn85s\" (UID: \"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.701303 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.701274    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:49.881106 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.879673    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"]
Jul 10 04:20:49.886461 ip-10-0-140-167 kubenswrapper[2561]: W0710 04:20:49.885817    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4cbf92da_02e7_4c75_9e9c_715d0d05f0ff.slice/crio-02f23a6f3720a1ab947464b242719b5bc25c25572d9a667df1d3ff39dbe27dba WatchSource:0}: Error finding container 02f23a6f3720a1ab947464b242719b5bc25c25572d9a667df1d3ff39dbe27dba: Status 404 returned error can't find the container with id 02f23a6f3720a1ab947464b242719b5bc25c25572d9a667df1d3ff39dbe27dba
Jul 10 04:20:49.986727 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:49.986694    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-4b5xs_05b51b6d-8448-4e78-9231-292041790e0f/volume-data-source-validator/0.log"
Jul 10 04:20:50.020809 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:50.020773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s" event={"ID":"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff","Type":"ContainerStarted","Data":"66ee1eeb416a5079deb27d6cad00fd7f7ae789918260a6cfacf7032a90363b2e"}
Jul 10 04:20:50.021075 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:50.021052    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s" event={"ID":"4cbf92da-02e7-4c75-9e9c-715d0d05f0ff","Type":"ContainerStarted","Data":"02f23a6f3720a1ab947464b242719b5bc25c25572d9a667df1d3ff39dbe27dba"}
Jul 10 04:20:50.021215 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:50.021202    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:50.038305 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:50.038256    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s" podStartSLOduration=1.038240256 podStartE2EDuration="1.038240256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 04:20:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:20:50.035399858 +0000 UTC m=+2911.526395757" watchObservedRunningTime="2026-07-10 04:20:50.038240256 +0000 UTC m=+2911.529236129"
Jul 10 04:20:51.025777 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:51.025740    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4rqhk_0843417a-dbe0-45e1-96c6-4510eb4d74ef/dns/0.log"
Jul 10 04:20:51.044718 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:51.044691    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4rqhk_0843417a-dbe0-45e1-96c6-4510eb4d74ef/kube-rbac-proxy/0.log"
Jul 10 04:20:51.133460 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:51.133428    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gz9tm_7aec8603-2c61-4c91-936d-728b9f3c641f/dns-node-resolver/0.log"
Jul 10 04:20:51.670130 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:51.670093    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4h4m7_4399c5c6-6b1b-4c30-bed8-d7ae494e282b/node-ca/0.log"
Jul 10 04:20:52.776552 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:52.776514    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d9645bfdd-gmc4w_dfca196d-b171-4d6c-8989-91ff58a6ffd5/kube-auth-proxy/0.log"
Jul 10 04:20:52.865459 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:52.865421    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-rnqxm_11c1301b-ab4e-490b-8012-e973bd2b00be/payload-pre-processing/0.log"
Jul 10 04:20:52.898066 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:52.898018    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ldtg2_fb1e2f13-1834-43a2-8f83-786caa86eaa2/payload-processing/0.log"
Jul 10 04:20:53.603787 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:53.603752    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m9dtr_39488c5e-12b6-4a76-9383-28aa9c74d14c/serve-healthcheck-canary/0.log"
Jul 10 04:20:54.066700 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:54.066579    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-l22sz_b858fbad-d7b2-4b25-b556-c852059bdef7/insights-operator/0.log"
Jul 10 04:20:54.069119 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:54.069094    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-l22sz_b858fbad-d7b2-4b25-b556-c852059bdef7/insights-operator/1.log"
Jul 10 04:20:54.156043 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:54.156015    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tnvwg_a2b5f043-d820-4234-a832-021b63224ee9/kube-rbac-proxy/0.log"
Jul 10 04:20:54.175508 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:54.175484    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tnvwg_a2b5f043-d820-4234-a832-021b63224ee9/exporter/0.log"
Jul 10 04:20:54.195663 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:54.195627    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tnvwg_a2b5f043-d820-4234-a832-021b63224ee9/extractor/0.log"
Jul 10 04:20:56.612137 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:56.612100    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f864f4b59-9h9qh_dc45a6d4-dac9-497f-91b9-873e433b0dde/manager/0.log"
Jul 10 04:20:56.765913 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:56.765884    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d676bbf4b-grr6m_822c49c8-42d2-42f5-9684-b21afcdc591f/manager/0.log"
Jul 10 04:20:56.784697 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:56.784670    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d676bbf4b-grr6m_822c49c8-42d2-42f5-9684-b21afcdc591f/copy-manifests/0.log"
Jul 10 04:20:57.046394 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:57.046306    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-zn85s"
Jul 10 04:20:58.101442 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:58.101413    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6549b595fc-klrpm_9df87cbb-2dab-483b-9ac5-d7ec4cb53fc4/manager/0.log"
Jul 10 04:20:59.356958 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:59.356926    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d9645bfdd-gmc4w_dfca196d-b171-4d6c-8989-91ff58a6ffd5/kube-auth-proxy/0.log"
Jul 10 04:20:59.426469 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:59.425563    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-rnqxm_11c1301b-ab4e-490b-8012-e973bd2b00be/payload-pre-processing/0.log"
Jul 10 04:20:59.448299 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:20:59.448270    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ldtg2_fb1e2f13-1834-43a2-8f83-786caa86eaa2/payload-processing/0.log"
Jul 10 04:21:00.807880 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:00.807846    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d9645bfdd-gmc4w_dfca196d-b171-4d6c-8989-91ff58a6ffd5/kube-auth-proxy/0.log"
Jul 10 04:21:00.881687 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:00.881546    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-rnqxm_11c1301b-ab4e-490b-8012-e973bd2b00be/payload-pre-processing/0.log"
Jul 10 04:21:00.912098 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:00.911955    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ldtg2_fb1e2f13-1834-43a2-8f83-786caa86eaa2/payload-processing/0.log"
Jul 10 04:21:03.353892 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:03.353865    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-xtcdb_fbb60d0f-3458-4983-baa7-905512ac72d8/migrator/0.log"
Jul 10 04:21:03.373669 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:03.373606    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-xtcdb_fbb60d0f-3458-4983-baa7-905512ac72d8/graceful-termination/0.log"
Jul 10 04:21:03.717174 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:03.717087    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-j6bzq_6c312753-450d-4a92-822e-4638a7eafc6f/kube-storage-version-migrator-operator/1.log"
Jul 10 04:21:03.717994 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:03.717975    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-j6bzq_6c312753-450d-4a92-822e-4638a7eafc6f/kube-storage-version-migrator-operator/0.log"
Jul 10 04:21:04.924798 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:04.924758    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5st2k_c23b127a-0ca0-4d4f-9de5-ef99040a7805/kube-multus-additional-cni-plugins/0.log"
Jul 10 04:21:04.944083 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:04.944057    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5st2k_c23b127a-0ca0-4d4f-9de5-ef99040a7805/egress-router-binary-copy/0.log"
Jul 10 04:21:04.963778 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:04.963745    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5st2k_c23b127a-0ca0-4d4f-9de5-ef99040a7805/cni-plugins/0.log"
Jul 10 04:21:04.983690 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:04.983666    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5st2k_c23b127a-0ca0-4d4f-9de5-ef99040a7805/bond-cni-plugin/0.log"
Jul 10 04:21:05.003429 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:05.003400    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5st2k_c23b127a-0ca0-4d4f-9de5-ef99040a7805/routeoverride-cni/0.log"
Jul 10 04:21:05.023309 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:05.023283    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5st2k_c23b127a-0ca0-4d4f-9de5-ef99040a7805/whereabouts-cni-bincopy/0.log"
Jul 10 04:21:05.042499 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:05.042471    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5st2k_c23b127a-0ca0-4d4f-9de5-ef99040a7805/whereabouts-cni/0.log"
Jul 10 04:21:05.264161 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:05.264076    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rf6sd_7ae35170-0a86-42c1-9c56-120f2281b128/kube-multus/0.log"
Jul 10 04:21:05.372480 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:05.372454    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p7l8g_776389fa-994a-4f3b-bcfb-48c486232e9b/network-metrics-daemon/0.log"
Jul 10 04:21:05.389230 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:05.389193    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p7l8g_776389fa-994a-4f3b-bcfb-48c486232e9b/kube-rbac-proxy/0.log"
Jul 10 04:21:06.455344 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:06.455249    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knklw_f6dacccf-f51b-4d6b-876a-90e1bdacaa97/ovn-controller/0.log"
Jul 10 04:21:06.470352 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:06.470323    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knklw_f6dacccf-f51b-4d6b-876a-90e1bdacaa97/ovn-acl-logging/0.log"
Jul 10 04:21:06.493317 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:06.493287    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knklw_f6dacccf-f51b-4d6b-876a-90e1bdacaa97/kube-rbac-proxy-node/0.log"
Jul 10 04:21:06.511622 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:06.511589    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knklw_f6dacccf-f51b-4d6b-876a-90e1bdacaa97/kube-rbac-proxy-ovn-metrics/0.log"
Jul 10 04:21:06.527154 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:06.527130    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knklw_f6dacccf-f51b-4d6b-876a-90e1bdacaa97/northd/0.log"
Jul 10 04:21:06.545744 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:06.545719    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knklw_f6dacccf-f51b-4d6b-876a-90e1bdacaa97/nbdb/0.log"
Jul 10 04:21:06.565144 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:06.565114    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knklw_f6dacccf-f51b-4d6b-876a-90e1bdacaa97/sbdb/0.log"
Jul 10 04:21:06.642421 ip-10-0-140-167 kubenswrapper[2561]: I0710 04:21:06.642387    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knklw_f6dacccf-f51b-4d6b-876a-90e1bdacaa97/ovnkube-controller/0.log"